2024-09-11T21:51:02.715 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T21:51:02.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T21:51:02.826 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901413' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEY2mRPFwrE7cDNRg/o+JI1ePJpd0ouBpaN6WpMhVR5mswxVSBrxRmMjC7qtDK3Te+cUeo3a31AZxi3e5wN3KH8= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOtPO6HK5vKiu7RUFTVqlVHPY2iNg6MBkGQVQcSLHmkEd0gkZ9gB0Pd6rkp016TTb7y22ycm+8DGiZMjl5Aor4= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQXSMhskbm7u79HFES+AhFiZ4oyKaw3AhmW2XirJsMXMBmQ95VHXzxI6e5SHfT0o5bz0LxwAqgC5NxugC2virM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T21:51:02.826 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T21:51:02.827 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T21:51:02.827 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T21:51:02.828 INFO:teuthology.task.internal:Checking packages... 2024-09-11T21:51:02.850 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T21:51:02.850 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T21:51:02.850 INFO:teuthology.packaging:ref: None 2024-09-11T21:51:02.850 INFO:teuthology.packaging:tag: None 2024-09-11T21:51:02.851 INFO:teuthology.packaging:branch: squid 2024-09-11T21:51:02.851 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:51:02.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T21:51:03.060 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T21:51:03.062 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T21:51:03.067 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T21:51:03.068 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T21:51:03.084 INFO:teuthology.task.internal:Saving configuration 2024-09-11T21:51:03.101 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T21:51:03.108 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T21:51:03.132 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:45:58.445730', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEY2mRPFwrE7cDNRg/o+JI1ePJpd0ouBpaN6WpMhVR5mswxVSBrxRmMjC7qtDK3Te+cUeo3a31AZxi3e5wN3KH8='} 2024-09-11T21:51:03.154 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:45:58.447483', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOtPO6HK5vKiu7RUFTVqlVHPY2iNg6MBkGQVQcSLHmkEd0gkZ9gB0Pd6rkp016TTb7y22ycm+8DGiZMjl5Aor4='} 2024-09-11T21:51:03.177 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:45:58.443271', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQXSMhskbm7u79HFES+AhFiZ4oyKaw3AhmW2XirJsMXMBmQ95VHXzxI6e5SHfT0o5bz0LxwAqgC5NxugC2virM='} 2024-09-11T21:51:03.177 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T21:51:03.183 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T21:51:03.183 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T21:51:03.183 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T21:51:03.183 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T21:51:03.265 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0d21953be0>, signals=[15]) 2024-09-11T21:51:03.265 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T21:51:03.270 INFO:teuthology.task.internal:Opening connections... 2024-09-11T21:51:03.270 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T21:51:03.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:51:03.352 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T21:51:03.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:51:03.426 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-09-11T21:51:03.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:51:03.500 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T21:51:03.507 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-11T21:51:03.527 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-11T21:51:03.527 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-11T21:51:03.584 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-11T21:51:03.584 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-11T21:51:03.584 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-11T21:51:03.584 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-11T21:51:03.584 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-11T21:51:03.584 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-11T21:51:03.584 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T21:51:03.585 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-11T21:51:03.585 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-11T21:51:03.585 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T21:51:03.585 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-11T21:51:03.585 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T21:51:03.585 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T21:51:03.585 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T21:51:03.586 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-11T21:51:03.606 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-11T21:51:03.624 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-11T21:51:03.625 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-11T21:51:03.682 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-09-11T21:51:03.682 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-09-11T21:51:03.682 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-09-11T21:51:03.682 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-09-11T21:51:03.682 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-09-11T21:51:03.682 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T21:51:03.683 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T21:51:03.683 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-11T21:51:03.706 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-09-11T21:51:03.724 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-09-11T21:51:03.724 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-09-11T21:51:03.782 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-09-11T21:51:03.782 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-09-11T21:51:03.782 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-09-11T21:51:03.782 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T21:51:03.783 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T21:51:03.784 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-09-11T21:51:03.805 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T21:51:03.815 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T21:51:03.820 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T21:51:03.821 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-11T21:51:03.823 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-11T21:51:03.825 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-09-11T21:51:03.842 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T21:51:03.848 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T21:51:03.848 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-11T21:51:03.880 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-11T21:51:03.883 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-09-11T21:51:03.976 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T21:51:04.046 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T21:51:04.059 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T21:51:04.059 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T21:51:04.060 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T21:51:04.060 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T21:51:04.060 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T21:51:04.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:51:04.078 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-11T21:51:04.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:51:04.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:51:04.079 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-09-11T21:51:04.080 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-11T21:51:04.135 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-503.el9.x86_64 2024-09-11T21:51:04.135 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-503.el9.x86_64 2024-09-11T21:51:04.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-09-11T21:51:04.136 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-503.el9.x86_64 2024-09-11T21:51:04.137 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-503.el9.x86_64 2024-09-11T21:51:04.137 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-09-11T21:51:04.138 INFO:teuthology.orchestra.run.smithi153.stdout:5.14.0-503.el9.x86_64 2024-09-11T21:51:04.138 INFO:teuthology.task.kernel:Running kernel on smithi153: 5.14.0-503.el9.x86_64 2024-09-11T21:51:04.138 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum install -y kernel 2024-09-11T21:51:06.521 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T21:51:06.603 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-11T21:51:08.304 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-11T21:51:08.572 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-09-11T21:51:10.330 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-11T21:51:10.397 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 20 MB 00:02 2024-09-11T21:51:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.5 MB 00:00 2024-09-11T21:51:15.629 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.5 MB 00:03 2024-09-11T21:51:15.922 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-11T21:51:16.807 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-11T21:51:17.154 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-11T21:51:17.464 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-11T21:51:17.736 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T21:51:18.200 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-11T21:51:18.402 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T21:51:23.192 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-11T21:51:23.614 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-11T21:51:23.799 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T21:51:25.215 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T21:51:25.215 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T21:51:25.302 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-11T21:51:25.304 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T21:51:25.305 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T21:51:25.306 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-11T21:51:25.306 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T21:51:25.306 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-09-11T21:51:25.306 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T21:51:25.306 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 79 M 2024-09-11T21:51:25.306 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 126 M 2024-09-11T21:51:25.306 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-11T21:51:25.619 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 79 kB/s | 23 kB 00:00 2024-09-11T21:51:25.687 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T21:51:25.687 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T21:51:25.786 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T21:51:25.795 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T21:51:25.796 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T21:51:25.797 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T21:51:25.797 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-11T21:51:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T21:51:25.797 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 79 M 2024-09-11T21:51:25.797 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 126 M 2024-09-11T21:51:25.797 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T21:51:25.813 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T21:51:25.814 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T21:51:25.905 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-09-11T21:51:25.915 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-09-11T21:51:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-09-11T21:51:25.915 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-09-11T21:51:25.915 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-09-11T21:51:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T21:51:25.915 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout:Install 4 Packages 2024-09-11T21:51:25.916 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T21:51:25.917 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 79 M 2024-09-11T21:51:25.917 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 126 M 2024-09-11T21:51:25.917 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-09-11T21:51:26.059 INFO:teuthology.orchestra.run.smithi153.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 187 kB/s | 23 kB 00:00 2024-09-11T21:51:26.340 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 79 kB/s | 23 kB 00:00 2024-09-11T21:51:27.534 INFO:teuthology.orchestra.run.smithi153.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 9.8 MB/s | 16 MB 00:01 2024-09-11T21:51:28.175 INFO:teuthology.orchestra.run.smithi153.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-09-11T21:51:34.125 INFO:teuthology.orchestra.run.smithi153.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.5 MB/s | 29 MB 00:08 2024-09-11T21:51:34.125 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:51:34.125 INFO:teuthology.orchestra.run.smithi153.stdout:Total 9.6 MB/s | 79 MB 00:08 2024-09-11T21:51:34.810 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-09-11T21:51:35.162 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-09-11T21:51:35.162 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-09-11T21:51:36.108 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-09-11T21:51:36.108 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-09-11T21:51:38.569 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-09-11T21:51:39.303 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:51:39.394 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:51:41.804 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-11T21:51:41.809 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 972 kB/s | 16 MB 00:16 2024-09-11T21:51:42.212 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:51:42.346 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:51:47.147 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:51:47.246 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:51:52.076 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:51:56.258 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 954 kB/s | 29 MB 00:30 2024-09-11T21:52:00.709 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 851 kB/s | 29 MB 00:34 2024-09-11T21:52:01.616 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 975 kB/s | 35 MB 00:36 2024-09-11T21:52:01.616 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:52:01.617 INFO:teuthology.orchestra.run.smithi110.stdout:Total 2.2 MB/s | 79 MB 00:36 2024-09-11T21:52:01.942 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 985 kB/s | 35 MB 00:35 2024-09-11T21:52:01.943 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:52:01.943 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.2 MB/s | 79 MB 00:36 2024-09-11T21:52:02.296 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-11T21:52:02.617 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T21:52:02.664 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-11T21:52:02.664 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-11T21:52:02.950 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T21:52:02.951 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T21:52:03.602 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-11T21:52:03.602 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-11T21:52:03.894 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T21:52:03.894 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T21:52:06.067 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-11T21:52:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T21:52:06.737 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:52:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:52:07.075 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:52:07.150 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:52:09.619 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:52:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:52:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:52:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:52:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:52:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:14.915 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:52:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:19.496 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:19.858 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:30.424 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:30.464 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:52:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:52:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:52:31.690 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:31.690 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T21:52:31.690 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-09-11T21:52:31.690 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:31.691 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T21:52:31.691 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T21:52:31.691 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T21:52:31.691 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T21:52:31.691 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-09-11T21:52:32.021 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:52:32.148 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:32.149 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:52:32.149 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T21:52:32.149 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T21:52:32.150 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T21:52:32.150 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-11T21:52:32.150 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi153.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T21:52:32.150 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:52:32.208 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:32.209 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:52:32.209 DEBUG:teuthology.orchestra.run.smithi153:> sudo rpm -qi grub2-tools 2024-09-11T21:52:32.244 INFO:teuthology.orchestra.run.smithi153.stdout:Name : grub2-tools 2024-09-11T21:52:32.244 INFO:teuthology.orchestra.run.smithi153.stdout:Epoch : 1 2024-09-11T21:52:32.244 INFO:teuthology.orchestra.run.smithi153.stdout:Version : 2.06 2024-09-11T21:52:32.244 INFO:teuthology.orchestra.run.smithi153.stdout:Release : 82.el9 2024-09-11T21:52:32.244 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: x86_64 2024-09-11T21:52:32.244 INFO:teuthology.orchestra.run.smithi153.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T21:52:32.244 INFO:teuthology.orchestra.run.smithi153.stdout:Group : Unspecified 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Size : 8274670 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:License : GPLv3+ 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Packager : builder@centos.org 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Vendor : CentOS 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Summary : Support tools for GRUB. 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout:Description : 2024-09-11T21:52:32.245 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T21:52:32.246 INFO:teuthology.orchestra.run.smithi153.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T21:52:32.246 INFO:teuthology.orchestra.run.smithi153.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T21:52:32.246 INFO:teuthology.orchestra.run.smithi153.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T21:52:32.246 INFO:teuthology.orchestra.run.smithi153.stdout:hardware devices. 2024-09-11T21:52:32.246 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T21:52:32.246 INFO:teuthology.orchestra.run.smithi153.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T21:52:32.247 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T21:52:32.247 INFO:teuthology.task.kernel:Updating grub on smithi153 to boot 5.14.0-505.el9.x86_64 2024-09-11T21:52:32.247 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T21:52:32.925 INFO:teuthology.orchestra.run.smithi153.stderr:Generating grub configuration file ... 2024-09-11T21:52:34.253 INFO:teuthology.orchestra.run.smithi153.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T21:52:34.282 INFO:teuthology.orchestra.run.smithi153.stderr:done 2024-09-11T21:52:34.285 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2024-09-11T21:52:34.301 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.Nf1VEv2P0h 2024-09-11T21:52:34.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Nf1VEv2P0h 2024-09-11T21:52:34.367 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.Nf1VEv2P0h 2024-09-11T21:52:34.514 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.Nf1VEv2P0h is 6KB 2024-09-11T21:52:34.564 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.Nf1VEv2P0h 2024-09-11T21:52:34.579 DEBUG:teuthology.orchestra.run.smithi153:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T21:52:34.642 INFO:teuthology.orchestra.run.smithi153.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T21:52:34.642 INFO:teuthology.orchestra.run.smithi153.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T21:52:34.642 INFO:teuthology.orchestra.run.smithi153.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T21:52:34.642 INFO:teuthology.orchestra.run.smithi153.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T21:52:34.644 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T21:52:34.797 DEBUG:teuthology.orchestra.run.smithi153:> sudo shutdown -r now 2024-09-11T21:52:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:57.784 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:57.950 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:57.950 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:52:57.950 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:52:57.950 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:52:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:52:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:52:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:52:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T21:52:58.793 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-11T21:52:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T21:52:58.794 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-11T21:52:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T21:52:58.859 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T21:52:59.164 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:52:59.175 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:52:59.249 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:59.250 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.250 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.250 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T21:52:59.250 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T21:52:59.250 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T21:52:59.250 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T21:52:59.250 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:52:59.254 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:59.255 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.255 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.255 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T21:52:59.255 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T21:52:59.255 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T21:52:59.255 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T21:52:59.255 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:52:59.309 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:59.309 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo rpm -qi grub2-tools 2024-09-11T21:52:59.313 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:52:59.314 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-09-11T21:52:59.341 INFO:teuthology.orchestra.run.smithi110.stdout:Name : grub2-tools 2024-09-11T21:52:59.341 INFO:teuthology.orchestra.run.smithi110.stdout:Epoch : 1 2024-09-11T21:52:59.341 INFO:teuthology.orchestra.run.smithi110.stdout:Version : 2.06 2024-09-11T21:52:59.341 INFO:teuthology.orchestra.run.smithi110.stdout:Release : 82.el9 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: x86_64 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Group : Unspecified 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Size : 8274670 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:License : GPLv3+ 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Packager : builder@centos.org 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Vendor : CentOS 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Summary : Support tools for GRUB. 2024-09-11T21:52:59.342 INFO:teuthology.orchestra.run.smithi110.stdout:Description : 2024-09-11T21:52:59.343 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T21:52:59.343 INFO:teuthology.orchestra.run.smithi110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T21:52:59.343 INFO:teuthology.orchestra.run.smithi110.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T21:52:59.343 INFO:teuthology.orchestra.run.smithi110.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T21:52:59.343 INFO:teuthology.orchestra.run.smithi110.stdout:hardware devices. 2024-09-11T21:52:59.343 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T21:52:59.343 INFO:teuthology.orchestra.run.smithi110.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T21:52:59.344 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T21:52:59.344 INFO:teuthology.task.kernel:Updating grub on smithi110 to boot 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.344 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T21:52:59.347 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-09-11T21:52:59.347 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-09-11T21:52:59.348 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-09-11T21:52:59.348 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 82.el9 2024-09-11T21:52:59.348 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-09-11T21:52:59.348 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T21:52:59.348 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-09-11T21:52:59.349 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8274670 2024-09-11T21:52:59.349 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-09-11T21:52:59.349 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T21:52:59.349 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T21:52:59.349 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T21:52:59.349 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T21:52:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T21:52:59.351 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-09-11T21:52:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T21:52:59.351 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T21:52:59.351 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T21:52:59.352 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-505.el9.x86_64 2024-09-11T21:52:59.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T21:52:59.971 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-11T21:53:00.035 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-11T21:53:01.299 INFO:teuthology.orchestra.run.smithi110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T21:53:01.328 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-11T21:53:01.338 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-11T21:53:01.353 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.fV71NJEScz 2024-09-11T21:53:01.354 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fV71NJEScz 2024-09-11T21:53:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T21:53:01.420 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.fV71NJEScz 2024-09-11T21:53:01.428 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-11T21:53:01.430 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-11T21:53:01.446 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.ex66Hw1BQp 2024-09-11T21:53:01.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ex66Hw1BQp 2024-09-11T21:53:01.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.ex66Hw1BQp 2024-09-11T21:53:01.557 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.fV71NJEScz is 6KB 2024-09-11T21:53:01.605 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.fV71NJEScz 2024-09-11T21:53:01.620 DEBUG:teuthology.orchestra.run.smithi110:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T21:53:01.668 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.ex66Hw1BQp is 6KB 2024-09-11T21:53:01.689 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.ex66Hw1BQp 2024-09-11T21:53:01.695 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T21:53:01.695 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T21:53:01.695 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T21:53:01.695 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T21:53:01.697 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T21:53:01.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T21:53:01.773 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T21:53:01.782 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T21:53:01.783 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T21:53:01.783 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T21:53:01.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T21:53:01.862 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-09-11T21:53:01.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-11T21:53:04.801 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-11T21:53:04.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-11T21:53:04.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:53:23.261 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-09-11T21:53:31.873 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T21:53:31.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T21:53:31.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:53:31.940 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T21:53:31.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T21:53:31.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:53:32.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-11T21:53:32.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:53:35.325 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-09-11T21:53:47.337 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-11T21:53:47.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:53:50.397 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-11T21:53:50.397 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-11T21:53:59.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T21:53:59.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:53:59.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T21:53:59.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:54:02.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-11T21:54:02.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-11T21:54:05.757 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-09-11T21:54:14.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T21:54:14.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:54:14.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T21:54:14.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:54:20.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-11T21:54:20.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:54:21.275 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-09-11T21:54:21.684 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-11T21:54:21.684 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-11T21:54:21.684 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-09-11T21:54:21.699 INFO:teuthology.orchestra.run.smithi153.stdout:5.14.0-505.el9.x86_64 2024-09-11T21:54:21.699 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T21:54:21.699 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T21:54:21.699 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T21:54:22.700 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-11T21:54:22.700 DEBUG:teuthology.orchestra.run.smithi153:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T21:54:22.778 INFO:teuthology.orchestra.run.smithi153.stdout:ttyS1 2024-09-11T21:54:22.808 DEBUG:teuthology.parallel:result is None 2024-09-11T21:54:32.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-11T21:54:32.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-11T21:54:47.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T21:54:47.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:54:47.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T21:54:47.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:54:48.217 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-11T21:54:48.244 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-11T21:54:48.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T21:54:48.702 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T21:54:48.702 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-11T21:54:48.725 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-505.el9.x86_64 2024-09-11T21:54:48.726 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T21:54:48.726 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T21:54:48.726 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T21:54:48.819 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T21:54:48.819 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T21:54:48.819 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-11T21:54:48.834 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-505.el9.x86_64 2024-09-11T21:54:48.834 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T21:54:48.834 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T21:54:48.834 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T21:54:49.726 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T21:54:49.727 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T21:54:49.792 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-09-11T21:54:49.815 DEBUG:teuthology.parallel:result is None 2024-09-11T21:54:49.835 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T21:54:49.835 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T21:54:49.908 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-11T21:54:49.930 DEBUG:teuthology.parallel:result is None 2024-09-11T21:54:49.931 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T21:54:49.938 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T21:54:49.938 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T21:54:49.940 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T21:54:49.943 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T21:54:49.963 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T21:54:49.981 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T21:54:50.060 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T21:54:50.060 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T21:54:50.062 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T21:54:50.065 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T21:54:50.100 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T21:54:50.106 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T21:54:50.106 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T21:54:50.130 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T21:54:50.136 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T21:54:50.162 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:54:50.169 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:54:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:54:50.180 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:54:50.180 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:54:50.190 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:54:50.192 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T21:54:50.199 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T21:54:50.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T21:54:50.218 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T21:54:50.220 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T21:54:50.249 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T21:54:50.258 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T21:54:50.258 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T21:54:50.283 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T21:54:50.286 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T21:54:50.306 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:54:50.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:54:50.429 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:54:50.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:54:50.550 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T21:54:50.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T21:54:50.615 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:54:50.659 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:54:50.709 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:54:50.790 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:54:50.830 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T21:54:50.830 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T21:54:50.894 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:54:50.939 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:54:50.991 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:54:51.074 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:54:51.118 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T21:54:51.118 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T21:54:51.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-11T21:54:51.189 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-11T21:54:51.191 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-09-11T21:54:51.232 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T21:54:51.233 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T21:54:51.274 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T21:54:51.748 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T21:54:51.756 INFO:teuthology.task.internal:Starting timer... 2024-09-11T21:54:51.757 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T21:54:51.767 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T21:54:51.775 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T21:54:51.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-11T21:54:51.847 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-11T21:54:51.849 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-09-11T21:54:51.917 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-09-11T21:54:51.919 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-09-11T21:54:51.992 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-09-11T21:54:51.995 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T21:54:51.995 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-11T21:54:52.023 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-11T21:54:52.023 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-09-11T21:54:52.051 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-09-11T21:54:52.052 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-09-11T21:54:52.089 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-09-11T21:54:52.090 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2024-09-11T21:54:52.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T21:54:52.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:54:52.120 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T21:54:52.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:54:52.148 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T21:54:52.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:54:52.178 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T21:54:52.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-11T21:54:52.219 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-09-11T21:54:52.256 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-09-11T21:54:52.284 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T21:54:52.294 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T21:54:52.298 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T21:54:52.431 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T21:54:52.442 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T21:54:52.443 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi110.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T22:00:52.087 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2024-09-11T22:00:52.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T22:00:52.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:00:52.176 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-11T22:00:52.257 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T22:00:52.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T22:00:52.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:00:52.331 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-11T22:00:52.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T22:00:52.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-11T22:00:52.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:00:52.495 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-09-11T22:00:52.589 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-11T22:00:52.590 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T22:00:52.600 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T22:00:52.600 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:00:52.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:00:52.604 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:00:52.604 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:00:52.606 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:00:52.606 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:00:52.633 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:00:52.647 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:00:52.652 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:00:52.661 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:00:52.671 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-09-11T22:00:52.676 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:00:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-11T22:00:52.682 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-09-11T22:00:52.694 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-09-11T22:00:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:00:52.710 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:00:52.713 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-09-11T22:00:52.726 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:00:52.727 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:00:52.742 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:00:52.773 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:00:52.777 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:00:52.777 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-11T22:00:52.777 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.777 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.777 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.777 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:00:52.786 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:00:52.786 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-11T22:00:52.786 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.786 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.786 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.786 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.799 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:00:52.802 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:00:52.803 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-11T22:00:52.803 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.803 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.803 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.803 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:00:52.804 INFO:teuthology.run_tasks:Running task install... 2024-09-11T22:00:52.812 DEBUG:teuthology.task.install:project ceph 2024-09-11T22:00:52.812 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T22:00:52.813 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T22:00:52.813 INFO:teuthology.task.install:Using flavor: default 2024-09-11T22:00:52.819 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T22:00:52.819 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T22:00:52.819 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:00:52.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:00:52.822 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:00:52.822 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:00:52.823 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:00:52.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:00:52.974 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:00:52.974 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:00:52.981 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:00:52.981 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:00:52.985 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:00:52.985 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:00:53.089 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:00:53.090 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:00:53.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:00:53.099 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:00:53.099 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:00:53.099 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:00:53.107 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:00:53.107 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:00:53.107 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:00:53.117 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:00:53.117 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:00:53.125 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:00:53.125 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:00:53.132 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:00:53.133 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:00:53.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:00:53.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:00:53.198 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:00:53.261 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:00:53.265 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:00:53.273 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:00:53.326 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-11T22:00:53.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-11T22:00:53.329 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-09-11T22:00:53.331 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-09-11T22:00:53.338 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-09-11T22:00:53.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-09-11T22:00:53.554 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-11T22:00:53.567 INFO:teuthology.orchestra.run.smithi153.stdout:45 files removed 2024-09-11T22:00:53.567 INFO:teuthology.orchestra.run.smithi110.stdout:45 files removed 2024-09-11T22:00:53.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:00:53.594 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:00:53.595 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:00:54.132 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-11T22:00:54.149 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-11T22:00:54.162 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-11T22:00:54.415 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T22:00:54.441 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-11T22:00:54.446 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T22:00:54.674 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-11T22:00:54.725 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-11T22:00:54.730 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-11T22:00:55.433 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T22:00:55.488 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T22:00:57.407 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-11T22:00:59.467 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 20 MB 00:02 2024-09-11T22:01:01.941 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.5 MB 00:00 2024-09-11T22:01:03.909 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-11T22:01:04.600 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 19 kB 00:01 2024-09-11T22:01:05.512 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T22:01:05.628 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-11T22:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T22:01:11.035 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-11T22:01:11.846 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T22:01:13.386 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:01:13.387 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:01:13.401 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:01:13.402 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:01:13.402 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:01:13.402 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:01:13.402 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:01:13.405 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:01:13.405 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:01:13.468 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-09-11T22:01:13.488 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-11T22:01:13.488 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-09-11T22:01:13.488 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-11T22:01:13.488 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-09-11T22:01:13.488 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:01:13.488 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:01:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:01:13.490 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:01:13.491 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:01:13.492 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:01:13.493 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:01:13.494 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:01:13.495 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:01:13.496 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:01:13.497 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:01:13.498 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:01:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:01:13.500 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:01:13.501 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:01:13.502 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout:Install 148 Packages 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 8 Packages 2024-09-11T22:01:13.503 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:01:13.504 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 226 M 2024-09-11T22:01:13.504 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-09-11T22:01:14.189 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:01:14.190 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:01:14.201 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:01:14.201 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:01:14.202 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:01:14.203 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:01:14.205 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:01:14.207 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:01:14.208 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:01:14.253 INFO:teuthology.orchestra.run.smithi153.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-11T22:01:14.270 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T22:01:14.284 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-11T22:01:14.284 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T22:01:14.284 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-11T22:01:14.284 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T22:01:14.284 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:01:14.284 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:01:14.284 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:01:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:01:14.286 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:01:14.287 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:01:14.288 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:01:14.289 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:01:14.290 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:01:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:01:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:01:14.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:01:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:01:14.295 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout:Install 148 Packages 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 8 Packages 2024-09-11T22:01:14.296 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:01:14.297 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 226 M 2024-09-11T22:01:14.297 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T22:01:14.486 INFO:teuthology.orchestra.run.smithi153.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-11T22:01:14.587 INFO:teuthology.orchestra.run.smithi153.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-11T22:01:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.1 MB/s | 145 kB 00:00 2024-09-11T22:01:14.712 INFO:teuthology.orchestra.run.smithi153.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 19 MB/s | 2.4 MB 00:00 2024-09-11T22:01:14.787 INFO:teuthology.orchestra.run.smithi153.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-11T22:01:14.914 INFO:teuthology.orchestra.run.smithi092.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2024-09-11T22:01:15.121 INFO:teuthology.orchestra.run.smithi153.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-11T22:01:15.329 INFO:teuthology.orchestra.run.smithi153.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 18 MB/s | 21 MB 00:01 2024-09-11T22:01:15.371 INFO:teuthology.orchestra.run.smithi153.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 608 kB/s | 25 kB 00:00 2024-09-11T22:01:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.9 MB/s | 897 kB 00:00 2024-09-11T22:01:15.489 INFO:teuthology.orchestra.run.smithi092.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.3 MB/s | 145 kB 00:00 2024-09-11T22:01:15.738 INFO:teuthology.orchestra.run.smithi153.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 18 MB/s | 17 MB 00:00 2024-09-11T22:01:15.805 INFO:teuthology.orchestra.run.smithi153.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-11T22:01:15.864 INFO:teuthology.orchestra.run.smithi153.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 464 kB/s | 27 kB 00:00 2024-09-11T22:01:15.922 INFO:teuthology.orchestra.run.smithi153.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 544 kB/s | 32 kB 00:00 2024-09-11T22:01:16.006 INFO:teuthology.orchestra.run.smithi153.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-11T22:01:16.040 INFO:teuthology.orchestra.run.smithi092.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-09-11T22:01:16.065 INFO:teuthology.orchestra.run.smithi153.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-11T22:01:16.090 INFO:teuthology.orchestra.run.smithi092.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 4.0 MB/s | 2.4 MB 00:00 2024-09-11T22:01:16.123 INFO:teuthology.orchestra.run.smithi153.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-11T22:01:16.207 INFO:teuthology.orchestra.run.smithi153.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-11T22:01:16.248 INFO:teuthology.orchestra.run.smithi092.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.3 MB/s | 1.5 MB 00:00 2024-09-11T22:01:16.499 INFO:teuthology.orchestra.run.smithi153.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-11T22:01:16.557 INFO:teuthology.orchestra.run.smithi153.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 775 kB/s | 45 kB 00:00 2024-09-11T22:01:16.674 INFO:teuthology.orchestra.run.smithi153.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.1 MB/s | 136 kB 00:00 2024-09-11T22:01:16.791 INFO:teuthology.orchestra.run.smithi153.stdout:(21/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 6.3 MB/s | 11 MB 00:01 2024-09-11T22:01:16.798 INFO:teuthology.orchestra.run.smithi092.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 6.6 MB/s | 4.7 MB 00:00 2024-09-11T22:01:16.816 INFO:teuthology.orchestra.run.smithi153.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.1 MB/s | 162 kB 00:00 2024-09-11T22:01:16.850 INFO:teuthology.orchestra.run.smithi153.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.3 MB/s | 321 kB 00:00 2024-09-11T22:01:16.892 INFO:teuthology.orchestra.run.smithi153.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.9 MB/s | 301 kB 00:00 2024-09-11T22:01:16.926 INFO:teuthology.orchestra.run.smithi153.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.3 MB/s | 100 kB 00:00 2024-09-11T22:01:16.959 INFO:teuthology.orchestra.run.smithi153.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-11T22:01:17.043 INFO:teuthology.orchestra.run.smithi153.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-11T22:01:17.118 INFO:teuthology.orchestra.run.smithi153.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 424 kB/s | 32 kB 00:00 2024-09-11T22:01:17.202 INFO:teuthology.orchestra.run.smithi153.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.7 MB/s | 146 kB 00:00 2024-09-11T22:01:17.486 INFO:teuthology.orchestra.run.smithi153.stdout:(30/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 13 MB/s | 3.6 MB 00:00 2024-09-11T22:01:17.557 INFO:teuthology.orchestra.run.smithi092.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 17 MB 00:01 2024-09-11T22:01:17.624 INFO:teuthology.orchestra.run.smithi092.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 379 kB/s | 25 kB 00:00 2024-09-11T22:01:17.695 INFO:teuthology.orchestra.run.smithi153.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 4.0 MB/s | 3.1 MB 00:00 2024-09-11T22:01:17.795 INFO:teuthology.orchestra.run.smithi153.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-11T22:01:17.824 INFO:teuthology.orchestra.run.smithi092.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 10 MB/s | 11 MB 00:01 2024-09-11T22:01:17.891 INFO:teuthology.orchestra.run.smithi092.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-11T22:01:17.975 INFO:teuthology.orchestra.run.smithi092.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 324 kB/s | 27 kB 00:00 2024-09-11T22:01:18.033 INFO:teuthology.orchestra.run.smithi092.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 545 kB/s | 32 kB 00:00 2024-09-11T22:01:18.108 INFO:teuthology.orchestra.run.smithi092.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-11T22:01:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-11T22:01:18.212 INFO:teuthology.orchestra.run.smithi153.stdout:(33/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 17 MB/s | 49 MB 00:02 2024-09-11T22:01:18.237 INFO:teuthology.orchestra.run.smithi153.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 111 kB/s | 49 kB 00:00 2024-09-11T22:01:18.251 INFO:teuthology.orchestra.run.smithi092.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-11T22:01:18.329 INFO:teuthology.orchestra.run.smithi153.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 8.8 MB/s | 7.4 MB 00:00 2024-09-11T22:01:18.334 INFO:teuthology.orchestra.run.smithi092.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-11T22:01:18.354 INFO:teuthology.orchestra.run.smithi153.stdout:(36/156): ceph-prometheus-alerts-19.1.1-298.gb5 116 kB/s | 16 kB 00:00 2024-09-11T22:01:18.379 INFO:teuthology.orchestra.run.smithi153.stdout:(37/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.0 MB/s | 285 kB 00:00 2024-09-11T22:01:18.446 INFO:teuthology.orchestra.run.smithi153.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 198 kB/s | 18 kB 00:00 2024-09-11T22:01:18.480 INFO:teuthology.orchestra.run.smithi153.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.0 MB/s | 765 kB 00:00 2024-09-11T22:01:18.547 INFO:teuthology.orchestra.run.smithi153.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 477 kB/s | 80 kB 00:00 2024-09-11T22:01:18.664 INFO:teuthology.orchestra.run.smithi153.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-11T22:01:18.689 INFO:teuthology.orchestra.run.smithi153.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 916 kB/s | 191 kB 00:00 2024-09-11T22:01:18.739 INFO:teuthology.orchestra.run.smithi153.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T22:01:18.760 INFO:teuthology.orchestra.run.smithi092.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-11T22:01:18.773 INFO:teuthology.orchestra.run.smithi153.stdout:(44/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-11T22:01:18.798 INFO:teuthology.orchestra.run.smithi153.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-11T22:01:18.801 INFO:teuthology.orchestra.run.smithi092.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 1.1 MB/s | 45 kB 00:00 2024-09-11T22:01:18.824 INFO:teuthology.orchestra.run.smithi153.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-11T22:01:18.866 INFO:teuthology.orchestra.run.smithi153.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-09-11T22:01:18.891 INFO:teuthology.orchestra.run.smithi153.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-11T22:01:19.008 INFO:teuthology.orchestra.run.smithi153.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 918 kB/s | 107 kB 00:00 2024-09-11T22:01:19.066 INFO:teuthology.orchestra.run.smithi153.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-11T22:01:19.092 INFO:teuthology.orchestra.run.smithi153.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-11T22:01:19.117 INFO:teuthology.orchestra.run.smithi153.stdout:(52/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-11T22:01:19.167 INFO:teuthology.orchestra.run.smithi153.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 33 kB 00:00 2024-09-11T22:01:19.201 INFO:teuthology.orchestra.run.smithi153.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-11T22:01:19.243 INFO:teuthology.orchestra.run.smithi153.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-09-11T22:01:19.260 INFO:teuthology.orchestra.run.smithi092.stdout:(20/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 30 MB/s | 49 MB 00:01 2024-09-11T22:01:19.268 INFO:teuthology.orchestra.run.smithi153.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-11T22:01:19.286 INFO:teuthology.orchestra.run.smithi092.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 281 kB/s | 136 kB 00:00 2024-09-11T22:01:19.311 INFO:teuthology.orchestra.run.smithi092.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-11T22:01:19.326 INFO:teuthology.orchestra.run.smithi153.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-09-11T22:01:19.336 INFO:teuthology.orchestra.run.smithi092.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-11T22:01:19.352 INFO:teuthology.orchestra.run.smithi153.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-11T22:01:19.361 INFO:teuthology.orchestra.run.smithi092.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-11T22:01:19.386 INFO:teuthology.orchestra.run.smithi092.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-11T22:01:19.412 INFO:teuthology.orchestra.run.smithi092.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T22:01:19.470 INFO:teuthology.orchestra.run.smithi092.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-11T22:01:19.477 INFO:teuthology.orchestra.run.smithi153.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-11T22:01:19.521 INFO:teuthology.orchestra.run.smithi092.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 637 kB/s | 32 kB 00:00 2024-09-11T22:01:19.527 INFO:teuthology.orchestra.run.smithi153.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-09-11T22:01:19.571 INFO:teuthology.orchestra.run.smithi092.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-11T22:01:19.586 INFO:teuthology.orchestra.run.smithi153.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-11T22:01:19.596 INFO:teuthology.orchestra.run.smithi092.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-11T22:01:19.645 INFO:teuthology.orchestra.run.smithi153.stdout:(62/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-09-11T22:01:19.737 INFO:teuthology.orchestra.run.smithi153.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-11T22:01:19.762 INFO:teuthology.orchestra.run.smithi153.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-11T22:01:19.771 INFO:teuthology.orchestra.run.smithi092.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 18 MB/s | 3.6 MB 00:00 2024-09-11T22:01:19.796 INFO:teuthology.orchestra.run.smithi153.stdout:(65/156): libtool-2.4.6-46.el9.x86_64.rpm 1.3 MB/s | 581 kB 00:00 2024-09-11T22:01:19.821 INFO:teuthology.orchestra.run.smithi153.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 460 kB/s | 38 kB 00:00 2024-09-11T22:01:19.880 INFO:teuthology.orchestra.run.smithi092.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 26 MB/s | 7.4 MB 00:00 2024-09-11T22:01:19.905 INFO:teuthology.orchestra.run.smithi092.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 1.8 MB/s | 249 kB 00:00 2024-09-11T22:01:19.931 INFO:teuthology.orchestra.run.smithi092.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 987 kB/s | 49 kB 00:00 2024-09-11T22:01:19.946 INFO:teuthology.orchestra.run.smithi153.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-11T22:01:19.956 INFO:teuthology.orchestra.run.smithi092.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 327 kB/s | 16 kB 00:00 2024-09-11T22:01:19.981 INFO:teuthology.orchestra.run.smithi092.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-11T22:01:20.015 INFO:teuthology.orchestra.run.smithi092.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 13 MB/s | 765 kB 00:00 2024-09-11T22:01:20.040 INFO:teuthology.orchestra.run.smithi092.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 311 kB/s | 18 kB 00:00 2024-09-11T22:01:20.071 INFO:teuthology.orchestra.run.smithi153.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-11T22:01:20.113 INFO:teuthology.orchestra.run.smithi153.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-11T22:01:20.214 INFO:teuthology.orchestra.run.smithi153.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 11 MB/s | 4.9 MB 00:00 2024-09-11T22:01:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 822 kB/s | 137 kB 00:00 2024-09-11T22:01:20.290 INFO:teuthology.orchestra.run.smithi092.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 290 kB/s | 80 kB 00:00 2024-09-11T22:01:20.314 INFO:teuthology.orchestra.run.smithi153.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi153.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-09-11T22:01:20.440 INFO:teuthology.orchestra.run.smithi153.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 924 kB/s | 61 kB 00:00 2024-09-11T22:01:20.532 INFO:teuthology.orchestra.run.smithi153.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-11T22:01:20.532 INFO:teuthology.orchestra.run.smithi092.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 790 kB/s | 191 kB 00:00 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi153.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-11T22:01:20.641 INFO:teuthology.orchestra.run.smithi092.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-11T22:01:20.699 INFO:teuthology.orchestra.run.smithi153.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-09-11T22:01:20.749 INFO:teuthology.orchestra.run.smithi092.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-11T22:01:20.757 INFO:teuthology.orchestra.run.smithi153.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-11T22:01:20.874 INFO:teuthology.orchestra.run.smithi092.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 853 kB/s | 106 kB 00:00 2024-09-11T22:01:20.882 INFO:teuthology.orchestra.run.smithi153.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-11T22:01:20.908 INFO:teuthology.orchestra.run.smithi153.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-11T22:01:20.950 INFO:teuthology.orchestra.run.smithi092.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-11T22:01:20.983 INFO:teuthology.orchestra.run.smithi153.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 6.6 MB/s | 5.8 MB 00:00 2024-09-11T22:01:21.008 INFO:teuthology.orchestra.run.smithi153.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 277 kB/s | 35 kB 00:00 2024-09-11T22:01:21.009 INFO:teuthology.orchestra.run.smithi092.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-11T22:01:21.092 INFO:teuthology.orchestra.run.smithi153.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-11T22:01:21.093 INFO:teuthology.orchestra.run.smithi092.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-09-11T22:01:21.117 INFO:teuthology.orchestra.run.smithi153.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-11T22:01:21.159 INFO:teuthology.orchestra.run.smithi092.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-09-11T22:01:21.193 INFO:teuthology.orchestra.run.smithi153.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 566 kB/s | 42 kB 00:00 2024-09-11T22:01:21.218 INFO:teuthology.orchestra.run.smithi153.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-09-11T22:01:21.301 INFO:teuthology.orchestra.run.smithi153.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-11T22:01:21.327 INFO:teuthology.orchestra.run.smithi153.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-11T22:01:21.394 INFO:teuthology.orchestra.run.smithi153.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-09-11T22:01:21.417 INFO:teuthology.orchestra.run.smithi092.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-11T22:01:21.419 INFO:teuthology.orchestra.run.smithi153.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-11T22:01:21.460 INFO:teuthology.orchestra.run.smithi092.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-11T22:01:21.493 INFO:teuthology.orchestra.run.smithi092.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 654 kB/s | 22 kB 00:00 2024-09-11T22:01:21.519 INFO:teuthology.orchestra.run.smithi153.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 429 kB/s | 54 kB 00:00 2024-09-11T22:01:21.527 INFO:teuthology.orchestra.run.smithi092.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-09-11T22:01:21.552 INFO:teuthology.orchestra.run.smithi092.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-11T22:01:21.569 INFO:teuthology.orchestra.run.smithi092.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 780 kB/s | 33 kB 00:00 2024-09-11T22:01:21.603 INFO:teuthology.orchestra.run.smithi092.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 551 kB/s | 18 kB 00:00 2024-09-11T22:01:21.636 INFO:teuthology.orchestra.run.smithi092.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-09-11T22:01:21.661 INFO:teuthology.orchestra.run.smithi153.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 294 kB/s | 42 kB 00:00 2024-09-11T22:01:21.686 INFO:teuthology.orchestra.run.smithi092.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-09-11T22:01:21.712 INFO:teuthology.orchestra.run.smithi092.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-11T22:01:21.728 INFO:teuthology.orchestra.run.smithi153.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 665 kB/s | 44 kB 00:00 2024-09-11T22:01:21.729 INFO:teuthology.orchestra.run.smithi092.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-11T22:01:21.779 INFO:teuthology.orchestra.run.smithi092.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-11T22:01:21.804 INFO:teuthology.orchestra.run.smithi092.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-11T22:01:21.830 INFO:teuthology.orchestra.run.smithi092.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-11T22:01:21.855 INFO:teuthology.orchestra.run.smithi092.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-11T22:01:21.889 INFO:teuthology.orchestra.run.smithi092.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-11T22:01:21.912 INFO:teuthology.orchestra.run.smithi153.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-11T22:01:21.914 INFO:teuthology.orchestra.run.smithi092.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-11T22:01:21.939 INFO:teuthology.orchestra.run.smithi092.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 771 kB/s | 38 kB 00:00 2024-09-11T22:01:22.004 INFO:teuthology.orchestra.run.smithi153.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.7 MB/s | 5.1 MB 00:01 2024-09-11T22:01:22.029 INFO:teuthology.orchestra.run.smithi153.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-11T22:01:22.031 INFO:teuthology.orchestra.run.smithi092.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 42 MB/s | 4.9 MB 00:00 2024-09-11T22:01:22.079 INFO:teuthology.orchestra.run.smithi153.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 630 kB/s | 221 kB 00:00 2024-09-11T22:01:22.098 INFO:teuthology.orchestra.run.smithi092.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-11T22:01:22.105 INFO:teuthology.orchestra.run.smithi153.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-11T22:01:22.140 INFO:teuthology.orchestra.run.smithi092.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T22:01:22.155 INFO:teuthology.orchestra.run.smithi153.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-11T22:01:22.172 INFO:teuthology.orchestra.run.smithi153.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 628 kB/s | 58 kB 00:00 2024-09-11T22:01:22.182 INFO:teuthology.orchestra.run.smithi092.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-11T22:01:22.197 INFO:teuthology.orchestra.run.smithi153.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.0 MB/s | 96 kB 00:00 2024-09-11T22:01:22.207 INFO:teuthology.orchestra.run.smithi092.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-11T22:01:22.256 INFO:teuthology.orchestra.run.smithi153.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 538 kB/s | 45 kB 00:00 2024-09-11T22:01:22.266 INFO:teuthology.orchestra.run.smithi092.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-11T22:01:22.281 INFO:teuthology.orchestra.run.smithi153.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-11T22:01:22.339 INFO:teuthology.orchestra.run.smithi153.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-11T22:01:22.341 INFO:teuthology.orchestra.run.smithi092.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-11T22:01:22.365 INFO:teuthology.orchestra.run.smithi153.stdout:(105/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 304 kB/s | 64 kB 00:00 2024-09-11T22:01:22.366 INFO:teuthology.orchestra.run.smithi092.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-09-11T22:01:22.398 INFO:teuthology.orchestra.run.smithi153.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-09-11T22:01:22.408 INFO:teuthology.orchestra.run.smithi092.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-11T22:01:22.424 INFO:teuthology.orchestra.run.smithi153.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-09-11T22:01:22.449 INFO:teuthology.orchestra.run.smithi153.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T22:01:22.482 INFO:teuthology.orchestra.run.smithi153.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-11T22:01:22.483 INFO:teuthology.orchestra.run.smithi092.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-09-11T22:01:22.508 INFO:teuthology.orchestra.run.smithi153.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 829 kB/s | 49 kB 00:00 2024-09-11T22:01:22.509 INFO:teuthology.orchestra.run.smithi092.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-11T22:01:22.534 INFO:teuthology.orchestra.run.smithi092.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-11T22:01:22.541 INFO:teuthology.orchestra.run.smithi153.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.8 MB/s | 151 kB 00:00 2024-09-11T22:01:22.551 INFO:teuthology.orchestra.run.smithi092.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-09-11T22:01:22.568 INFO:teuthology.orchestra.run.smithi092.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-09-11T22:01:22.575 INFO:teuthology.orchestra.run.smithi153.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 731 kB/s | 67 kB 00:00 2024-09-11T22:01:22.593 INFO:teuthology.orchestra.run.smithi092.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-11T22:01:22.610 INFO:teuthology.orchestra.run.smithi092.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-09-11T22:01:22.625 INFO:teuthology.orchestra.run.smithi153.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-09-11T22:01:22.652 INFO:teuthology.orchestra.run.smithi092.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-11T22:01:22.659 INFO:teuthology.orchestra.run.smithi153.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-11T22:01:22.684 INFO:teuthology.orchestra.run.smithi153.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-09-11T22:01:22.709 INFO:teuthology.orchestra.run.smithi153.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-11T22:01:22.726 INFO:teuthology.orchestra.run.smithi153.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 337 kB/s | 14 kB 00:00 2024-09-11T22:01:22.727 INFO:teuthology.orchestra.run.smithi092.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-11T22:01:22.753 INFO:teuthology.orchestra.run.smithi092.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-11T22:01:22.760 INFO:teuthology.orchestra.run.smithi153.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T22:01:22.778 INFO:teuthology.orchestra.run.smithi092.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-11T22:01:22.794 INFO:teuthology.orchestra.run.smithi153.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-11T22:01:22.795 INFO:teuthology.orchestra.run.smithi092.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-11T22:01:22.820 INFO:teuthology.orchestra.run.smithi092.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-11T22:01:22.828 INFO:teuthology.orchestra.run.smithi153.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-11T22:01:22.846 INFO:teuthology.orchestra.run.smithi092.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-11T22:01:22.853 INFO:teuthology.orchestra.run.smithi153.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T22:01:22.871 INFO:teuthology.orchestra.run.smithi092.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-11T22:01:22.888 INFO:teuthology.orchestra.run.smithi092.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-11T22:01:22.905 INFO:teuthology.orchestra.run.smithi092.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-11T22:01:22.911 INFO:teuthology.orchestra.run.smithi153.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi153.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi092.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi153.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-11T22:01:22.972 INFO:teuthology.orchestra.run.smithi092.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-11T22:01:22.987 INFO:teuthology.orchestra.run.smithi153.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-11T22:01:23.013 INFO:teuthology.orchestra.run.smithi153.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-11T22:01:23.038 INFO:teuthology.orchestra.run.smithi153.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-11T22:01:23.063 INFO:teuthology.orchestra.run.smithi153.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-09-11T22:01:23.089 INFO:teuthology.orchestra.run.smithi153.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:01:23.114 INFO:teuthology.orchestra.run.smithi153.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T22:01:23.139 INFO:teuthology.orchestra.run.smithi153.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T22:01:23.172 INFO:teuthology.orchestra.run.smithi092.stdout:(94/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 2.5 MB/s | 21 MB 00:08 2024-09-11T22:01:23.181 INFO:teuthology.orchestra.run.smithi153.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-11T22:01:23.198 INFO:teuthology.orchestra.run.smithi092.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 980 kB/s | 221 kB 00:00 2024-09-11T22:01:23.206 INFO:teuthology.orchestra.run.smithi153.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-11T22:01:23.231 INFO:teuthology.orchestra.run.smithi092.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-11T22:01:23.248 INFO:teuthology.orchestra.run.smithi153.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-11T22:01:23.256 INFO:teuthology.orchestra.run.smithi092.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 430 kB/s | 36 kB 00:00 2024-09-11T22:01:23.273 INFO:teuthology.orchestra.run.smithi153.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-11T22:01:23.290 INFO:teuthology.orchestra.run.smithi092.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-11T22:01:23.299 INFO:teuthology.orchestra.run.smithi153.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-11T22:01:23.324 INFO:teuthology.orchestra.run.smithi153.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-11T22:01:23.349 INFO:teuthology.orchestra.run.smithi153.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T22:01:23.374 INFO:teuthology.orchestra.run.smithi153.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-11T22:01:23.400 INFO:teuthology.orchestra.run.smithi153.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-11T22:01:23.425 INFO:teuthology.orchestra.run.smithi153.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-11T22:01:23.449 INFO:teuthology.orchestra.run.smithi092.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-11T22:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T22:01:23.465 INFO:teuthology.orchestra.run.smithi092.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 277 kB/s | 58 kB 00:00 2024-09-11T22:01:23.475 INFO:teuthology.orchestra.run.smithi153.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T22:01:23.490 INFO:teuthology.orchestra.run.smithi092.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 478 kB/s | 96 kB 00:00 2024-09-11T22:01:23.507 INFO:teuthology.orchestra.run.smithi092.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-11T22:01:23.524 INFO:teuthology.orchestra.run.smithi092.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-09-11T22:01:23.526 INFO:teuthology.orchestra.run.smithi153.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-09-11T22:01:23.568 INFO:teuthology.orchestra.run.smithi153.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-09-11T22:01:23.591 INFO:teuthology.orchestra.run.smithi092.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-11T22:01:23.601 INFO:teuthology.orchestra.run.smithi153.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-11T22:01:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-11T22:01:23.626 INFO:teuthology.orchestra.run.smithi153.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-09-11T22:01:23.642 INFO:teuthology.orchestra.run.smithi092.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-11T22:01:23.669 INFO:teuthology.orchestra.run.smithi153.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T22:01:23.743 INFO:teuthology.orchestra.run.smithi092.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-11T22:01:23.784 INFO:teuthology.orchestra.run.smithi092.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 967 kB/s | 26 kB 00:00 2024-09-11T22:01:23.809 INFO:teuthology.orchestra.run.smithi092.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T22:01:23.835 INFO:teuthology.orchestra.run.smithi092.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T22:01:23.844 INFO:teuthology.orchestra.run.smithi153.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-11T22:01:23.860 INFO:teuthology.orchestra.run.smithi092.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.3 MB/s | 151 kB 00:00 2024-09-11T22:01:23.893 INFO:teuthology.orchestra.run.smithi092.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 25 MB/s | 839 kB 00:00 2024-09-11T22:01:23.927 INFO:teuthology.orchestra.run.smithi092.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-11T22:01:23.953 INFO:teuthology.orchestra.run.smithi092.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T22:01:23.961 INFO:teuthology.orchestra.run.smithi153.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.5 MB/s | 3.2 MB 00:00 2024-09-11T22:01:23.978 INFO:teuthology.orchestra.run.smithi092.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T22:01:24.003 INFO:teuthology.orchestra.run.smithi092.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-11T22:01:24.028 INFO:teuthology.orchestra.run.smithi092.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-11T22:01:24.062 INFO:teuthology.orchestra.run.smithi092.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-11T22:01:24.069 INFO:teuthology.orchestra.run.smithi153.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-11T22:01:24.104 INFO:teuthology.orchestra.run.smithi092.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-09-11T22:01:24.129 INFO:teuthology.orchestra.run.smithi092.stdout:(120/156): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-09-11T22:01:24.163 INFO:teuthology.orchestra.run.smithi092.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T22:01:24.188 INFO:teuthology.orchestra.run.smithi092.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 6.1 MB/s | 144 kB 00:00 2024-09-11T22:01:24.205 INFO:teuthology.orchestra.run.smithi092.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 648 kB/s | 11 kB 00:00 2024-09-11T22:01:24.220 INFO:teuthology.orchestra.run.smithi153.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 1.6 MB/s | 249 kB 00:00 2024-09-11T22:01:24.222 INFO:teuthology.orchestra.run.smithi092.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-11T22:01:24.239 INFO:teuthology.orchestra.run.smithi092.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-11T22:01:24.256 INFO:teuthology.orchestra.run.smithi092.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-11T22:01:24.281 INFO:teuthology.orchestra.run.smithi092.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-11T22:01:24.295 INFO:teuthology.orchestra.run.smithi153.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.7 MB/s | 2.3 MB 00:00 2024-09-11T22:01:24.340 INFO:teuthology.orchestra.run.smithi092.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-09-11T22:01:24.362 INFO:teuthology.orchestra.run.smithi153.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 3.5 MB/s | 1.8 MB 00:00 2024-09-11T22:01:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:01:24.398 INFO:teuthology.orchestra.run.smithi092.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-11T22:01:24.424 INFO:teuthology.orchestra.run.smithi092.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-11T22:01:24.449 INFO:teuthology.orchestra.run.smithi092.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-11T22:01:24.454 INFO:teuthology.orchestra.run.smithi153.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-11T22:01:24.474 INFO:teuthology.orchestra.run.smithi092.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-11T22:01:24.499 INFO:teuthology.orchestra.run.smithi092.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-11T22:01:24.533 INFO:teuthology.orchestra.run.smithi092.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-11T22:01:24.567 INFO:teuthology.orchestra.run.smithi092.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-09-11T22:01:24.592 INFO:teuthology.orchestra.run.smithi092.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-09-11T22:01:24.617 INFO:teuthology.orchestra.run.smithi092.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T22:01:24.643 INFO:teuthology.orchestra.run.smithi092.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-11T22:01:24.660 INFO:teuthology.orchestra.run.smithi092.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 853 kB/s | 36 kB 00:00 2024-09-11T22:01:24.685 INFO:teuthology.orchestra.run.smithi092.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-11T22:01:24.710 INFO:teuthology.orchestra.run.smithi092.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T22:01:24.752 INFO:teuthology.orchestra.run.smithi092.stdout:(143/156): protobuf-compiler-3.14.0-13.el9.x86_ 694 kB/s | 863 kB 00:01 2024-09-11T22:01:24.777 INFO:teuthology.orchestra.run.smithi092.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-11T22:01:24.811 INFO:teuthology.orchestra.run.smithi092.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.2 MB/s | 191 kB 00:00 2024-09-11T22:01:24.836 INFO:teuthology.orchestra.run.smithi092.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-11T22:01:24.878 INFO:teuthology.orchestra.run.smithi092.stdout:(147/156): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-09-11T22:01:24.929 INFO:teuthology.orchestra.run.smithi092.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T22:01:25.154 INFO:teuthology.orchestra.run.smithi092.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-11T22:01:25.204 INFO:teuthology.orchestra.run.smithi092.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.3 MB/s | 3.4 MB 00:00 2024-09-11T22:01:25.404 INFO:teuthology.orchestra.run.smithi092.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 204 kB/s | 41 kB 00:00 2024-09-11T22:01:25.471 INFO:teuthology.orchestra.run.smithi153.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-11T22:01:25.476 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:01:25.476 INFO:teuthology.orchestra.run.smithi153.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-11T22:01:25.654 INFO:teuthology.orchestra.run.smithi092.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 995 kB/s | 249 kB 00:00 2024-09-11T22:01:25.913 INFO:teuthology.orchestra.run.smithi092.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 2.4 MB/s | 2.3 MB 00:00 2024-09-11T22:01:25.988 INFO:teuthology.orchestra.run.smithi092.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 2.2 MB/s | 1.8 MB 00:00 2024-09-11T22:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 1.4 MB 00:00 2024-09-11T22:01:26.638 INFO:teuthology.orchestra.run.smithi092.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 2.4 MB 00:00 2024-09-11T22:01:26.644 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:01:26.644 INFO:teuthology.orchestra.run.smithi092.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-11T22:01:26.697 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-09-11T22:01:26.848 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-09-11T22:01:26.848 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-09-11T22:01:27.876 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T22:01:28.028 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T22:01:28.028 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T22:01:29.169 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-09-11T22:01:29.170 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-09-11T22:01:30.251 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T22:01:30.252 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T22:01:35.130 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 209 kB/s | 8.2 MB 00:40 2024-09-11T22:01:35.544 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-09-11T22:01:35.650 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:01:35.704 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:01:35.767 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:01:35.800 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:01:35.825 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:01:35.896 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:01:36.266 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:01:36.376 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:01:36.499 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:01:36.527 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:01:36.619 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:01:36.657 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:01:36.664 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T22:01:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:01:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:01:36.837 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:01:36.851 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:01:36.886 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:01:36.892 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:01:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:01:36.917 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:01:36.966 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:01:37.019 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:01:37.049 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:01:37.112 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:01:37.123 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-11T22:01:37.133 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:01:37.156 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:01:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:01:37.335 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:01:37.418 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:01:37.519 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:01:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:01:37.569 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:01:37.648 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:01:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:01:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:01:37.765 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:01:37.842 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:01:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:01:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:01:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:01:38.067 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:01:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:01:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:01:38.175 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:01:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:01:38.276 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:01:38.329 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:01:38.373 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:01:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:01:38.438 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:01:38.482 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:01:38.514 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:01:38.566 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:01:38.594 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:01:38.607 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:01:38.635 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:01:38.704 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:01:38.758 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:01:38.759 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:01:38.788 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:01:38.848 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:01:38.875 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:01:38.924 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:01:38.950 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:01:38.958 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:01:39.002 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:01:39.039 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:01:39.110 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:01:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:01:39.237 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:01:39.323 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:01:39.367 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:01:39.419 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:01:39.422 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:01:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:01:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:01:39.563 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:01:39.615 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:01:39.647 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:01:39.691 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:01:39.724 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:01:39.751 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:01:39.820 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:01:39.873 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:01:39.904 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:01:39.937 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:01:39.939 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:01:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:01:40.026 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:01:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:01:40.055 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:01:40.098 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:01:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:01:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:01:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:01:40.179 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:01:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:01:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:01:40.334 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:01:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:01:40.398 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:01:40.453 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:01:40.484 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:01:40.525 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:01:40.539 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:01:40.566 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:01:40.601 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:01:40.648 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:01:40.684 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:01:40.736 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:01:40.778 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:01:40.914 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:01:40.958 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:01:40.999 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:01:41.061 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:01:41.102 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:01:41.141 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:01:41.159 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:01:41.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:01:41.193 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:01:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:01:41.265 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:01:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:01:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:01:41.481 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:01:41.516 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:01:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:01:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:01:41.669 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:01:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:01:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:01:41.742 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:01:41.788 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:01:41.801 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:01:41.824 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:01:41.838 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:01:41.866 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:01:41.867 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:01:41.907 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:01:41.909 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:01:41.948 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:01:41.988 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:01:42.035 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:01:42.052 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:01:42.069 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:01:42.102 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:01:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:01:42.144 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:01:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:01:42.178 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:01:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:01:42.264 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:01:42.309 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:01:42.348 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:01:42.449 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-11T22:01:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:01:42.507 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:01:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:01:42.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:01:42.660 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:01:42.693 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:01:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:01:42.931 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:01:42.962 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:01:42.990 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:01:43.033 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:01:43.071 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:01:43.103 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:01:43.133 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:01:43.167 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:01:43.201 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:01:43.223 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:01:43.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:01:43.271 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:01:43.276 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:01:43.313 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:01:43.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:01:43.407 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:01:43.411 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:01:43.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:01:43.460 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:01:43.504 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:01:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:01:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:01:43.592 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:01:43.632 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:01:43.826 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:01:43.863 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:01:43.941 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:01:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:01:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:01:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:01:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:01:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:01:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:01:44.587 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:01:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:01:44.704 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:01:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:01:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:01:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:01:44.966 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 19 kB 00:01 2024-09-11T22:01:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:01:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:01:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:01:45.974 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:01:46.052 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:01:46.079 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:01:46.095 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-11T22:01:46.112 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:01:46.425 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:01:46.467 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:01:46.559 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:01:46.588 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:01:46.630 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:01:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:01:47.112 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:01:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:01:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:01:47.145 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:01:47.175 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:01:47.216 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:01:47.247 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:01:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:01:47.789 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:01:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:01:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:01:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:01:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:01:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:01:48.545 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:01:48.567 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:01:49.544 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:01:49.576 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:01:50.126 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:01:50.164 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:01:50.258 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:01:50.258 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:01:50.258 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:01:50.258 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:01:50.300 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:01:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:01:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:01:51.462 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:01:51.508 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:01:51.518 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-11T22:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:01:53.963 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:01:53.964 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:01:53.977 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:01:53.978 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:01:53.979 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:01:53.980 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:01:53.982 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:01:53.985 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:01:53.986 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:01:54.048 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-11T22:01:54.072 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-11T22:01:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-11T22:01:54.073 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-11T22:01:54.073 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-11T22:01:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:01:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:01:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:01:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:01:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:01:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:01:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:01:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:01:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:01:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:01:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:01:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:01:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:01:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:01:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:01:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:01:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:01:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:01:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:01:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:01:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:01:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:01:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-11T22:01:54.088 INFO:teuthology.orchestra.run.smithi110.stdout:Install 148 Packages 2024-09-11T22:01:54.089 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 8 Packages 2024-09-11T22:01:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:01:54.089 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 226 M 2024-09-11T22:01:54.089 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-11T22:01:54.578 INFO:teuthology.orchestra.run.smithi110.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-11T22:01:54.770 INFO:teuthology.orchestra.run.smithi110.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.6 MB/s | 897 kB 00:00 2024-09-11T22:01:54.837 INFO:teuthology.orchestra.run.smithi110.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-11T22:01:54.979 INFO:teuthology.orchestra.run.smithi110.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-11T22:01:55.088 INFO:teuthology.orchestra.run.smithi110.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-11T22:01:55.146 INFO:teuthology.orchestra.run.smithi110.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 7.7 MB/s | 2.4 MB 00:00 2024-09-11T22:01:55.297 INFO:teuthology.orchestra.run.smithi110.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 23 MB/s | 4.7 MB 00:00 2024-09-11T22:01:55.747 INFO:teuthology.orchestra.run.smithi110.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-11T22:01:55.881 INFO:teuthology.orchestra.run.smithi110.stdout:(9/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 18 MB/s | 11 MB 00:00 2024-09-11T22:01:55.906 INFO:teuthology.orchestra.run.smithi110.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 159 kB/s | 25 kB 00:00 2024-09-11T22:01:55.989 INFO:teuthology.orchestra.run.smithi110.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 228 kB 00:00 2024-09-11T22:01:56.073 INFO:teuthology.orchestra.run.smithi110.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 324 kB/s | 27 kB 00:00 2024-09-11T22:01:56.140 INFO:teuthology.orchestra.run.smithi110.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 476 kB/s | 32 kB 00:00 2024-09-11T22:01:56.291 INFO:teuthology.orchestra.run.smithi110.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.9 MB/s | 752 kB 00:00 2024-09-11T22:01:56.391 INFO:teuthology.orchestra.run.smithi110.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.6 MB/s | 163 kB 00:00 2024-09-11T22:01:56.458 INFO:teuthology.orchestra.run.smithi110.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-11T22:01:56.650 INFO:teuthology.orchestra.run.smithi110.stdout:(17/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 17 MB 00:01 2024-09-11T22:01:56.683 INFO:teuthology.orchestra.run.smithi110.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 2.2 MB/s | 506 kB 00:00 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi110.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 905 kB/s | 45 kB 00:00 2024-09-11T22:01:56.817 INFO:teuthology.orchestra.run.smithi110.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-11T22:01:56.893 INFO:teuthology.orchestra.run.smithi110.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 162 kB 00:00 2024-09-11T22:01:56.985 INFO:teuthology.orchestra.run.smithi110.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 321 kB 00:00 2024-09-11T22:01:57.093 INFO:teuthology.orchestra.run.smithi110.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-11T22:01:57.119 INFO:teuthology.orchestra.run.smithi110.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.2 MB/s | 301 kB 00:00 2024-09-11T22:01:57.144 INFO:teuthology.orchestra.run.smithi110.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-11T22:01:57.187 INFO:teuthology.orchestra.run.smithi110.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-11T22:01:57.254 INFO:teuthology.orchestra.run.smithi110.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-11T22:01:57.704 INFO:teuthology.orchestra.run.smithi110.stdout:(28/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 27 MB/s | 49 MB 00:01 2024-09-11T22:01:57.729 INFO:teuthology.orchestra.run.smithi110.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 67 kB/s | 32 kB 00:00 2024-09-11T22:01:57.771 INFO:teuthology.orchestra.run.smithi110.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.1 MB/s | 146 kB 00:00 2024-09-11T22:01:57.830 INFO:teuthology.orchestra.run.smithi110.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 4.5 MB/s | 3.1 MB 00:00 2024-09-11T22:01:57.964 INFO:teuthology.orchestra.run.smithi110.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 15 MB/s | 3.6 MB 00:00 2024-09-11T22:01:57.989 INFO:teuthology.orchestra.run.smithi110.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 1.5 MB/s | 249 kB 00:00 2024-09-11T22:01:58.014 INFO:teuthology.orchestra.run.smithi110.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 986 kB/s | 49 kB 00:00 2024-09-11T22:01:58.039 INFO:teuthology.orchestra.run.smithi110.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 328 kB/s | 16 kB 00:00 2024-09-11T22:01:58.073 INFO:teuthology.orchestra.run.smithi110.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-11T22:01:58.123 INFO:teuthology.orchestra.run.smithi110.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 366 kB/s | 18 kB 00:00 2024-09-11T22:01:58.165 INFO:teuthology.orchestra.run.smithi110.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.0 MB/s | 765 kB 00:00 2024-09-11T22:01:58.266 INFO:teuthology.orchestra.run.smithi110.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-11T22:01:58.307 INFO:teuthology.orchestra.run.smithi110.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 435 kB/s | 80 kB 00:00 2024-09-11T22:01:58.391 INFO:teuthology.orchestra.run.smithi110.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-11T22:01:58.450 INFO:teuthology.orchestra.run.smithi110.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-11T22:01:58.484 INFO:teuthology.orchestra.run.smithi110.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-11T22:01:58.517 INFO:teuthology.orchestra.run.smithi110.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-09-11T22:01:58.542 INFO:teuthology.orchestra.run.smithi110.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-11T22:01:58.593 INFO:teuthology.orchestra.run.smithi110.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-11T22:01:58.618 INFO:teuthology.orchestra.run.smithi110.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-09-11T22:01:58.643 INFO:teuthology.orchestra.run.smithi110.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-11T22:01:59.335 INFO:teuthology.orchestra.run.smithi110.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 154 kB/s | 107 kB 00:00 2024-09-11T22:01:59.594 INFO:teuthology.orchestra.run.smithi110.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 688 kB/s | 689 kB 00:01 2024-09-11T22:01:59.627 INFO:teuthology.orchestra.run.smithi110.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 935 kB/s | 943 kB 00:01 2024-09-11T22:01:59.869 INFO:teuthology.orchestra.run.smithi110.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-09-11T22:02:00.245 INFO:teuthology.orchestra.run.smithi110.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-09-11T22:02:00.454 INFO:teuthology.orchestra.run.smithi110.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 365 kB/s | 313 kB 00:00 2024-09-11T22:02:00.770 INFO:teuthology.orchestra.run.smithi110.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-09-11T22:02:00.954 INFO:teuthology.orchestra.run.smithi110.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-11T22:02:01.071 INFO:teuthology.orchestra.run.smithi110.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-11T22:02:01.571 INFO:teuthology.orchestra.run.smithi110.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2024-09-11T22:02:01.638 INFO:teuthology.orchestra.run.smithi110.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-09-11T22:02:02.040 INFO:teuthology.orchestra.run.smithi110.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 690 kB/s | 662 kB 00:00 2024-09-11T22:02:02.188 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-09-11T22:02:02.189 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:02.205 INFO:teuthology.orchestra.run.smithi110.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 43 kB/s | 24 kB 00:00 2024-09-11T22:02:02.366 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:02:02.381 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:02:02.382 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:02.382 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:02:02.382 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:02.514 INFO:teuthology.orchestra.run.smithi110.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 617 kB/s | 581 kB 00:00 2024-09-11T22:02:02.631 INFO:teuthology.orchestra.run.smithi110.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-09-11T22:02:02.998 INFO:teuthology.orchestra.run.smithi110.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 369 kB/s | 292 kB 00:00 2024-09-11T22:02:03.194 INFO:teuthology.orchestra.run.smithi110.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-11T22:02:03.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-11T22:02:03.300 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:03.342 INFO:teuthology.orchestra.run.smithi110.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 228 kB/s | 188 kB 00:00 2024-09-11T22:02:03.471 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:02:03.487 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:02:03.487 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:03.487 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:02:03.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:03.876 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:02:03.904 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:02:03.904 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:03.904 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:02:03.904 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:02:03.904 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:02:03.904 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:04.136 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:02:04.160 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:02:04.161 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:04.161 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:02:04.161 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:02:04.161 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:02:04.161 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:04.166 INFO:teuthology.orchestra.run.smithi110.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.2 MB/s | 4.9 MB 00:01 2024-09-11T22:02:04.416 INFO:teuthology.orchestra.run.smithi110.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-11T22:02:04.499 INFO:teuthology.orchestra.run.smithi110.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-09-11T22:02:04.595 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:02:04.619 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:02:04.619 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:04.619 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:02:04.619 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:02:04.619 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:02:04.620 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:04.734 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:02:04.782 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:02:04.837 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:02:04.957 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:02:04.984 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:02:04.984 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:04.984 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:02:04.984 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:02:04.984 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:02:04.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:05.019 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:02:05.079 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:02:05.141 INFO:teuthology.orchestra.run.smithi110.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 192 kB/s | 137 kB 00:00 2024-09-11T22:02:05.155 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:02:05.183 INFO:teuthology.orchestra.run.smithi110.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 895 kB/s | 910 kB 00:01 2024-09-11T22:02:05.216 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:02:05.239 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:02:05.239 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:05.239 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:02:05.239 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:02:05.239 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:02:05.240 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:05.244 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:02:05.374 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:02:05.421 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:02:05.453 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:02:05.481 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:02:05.650 INFO:teuthology.orchestra.run.smithi110.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-11T22:02:05.675 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:02:05.698 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:02:05.698 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:05.698 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:02:05.698 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:02:05.698 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:02:05.699 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi110.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 131 kB/s | 87 kB 00:00 2024-09-11T22:02:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:02:05.872 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:02:05.904 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:02:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:02:05.935 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:02:05.997 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:02:06.036 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:02:06.100 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:02:06.101 INFO:teuthology.orchestra.run.smithi110.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-11T22:02:06.150 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:02:06.219 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:02:06.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:02:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:02:06.500 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:02:06.509 INFO:teuthology.orchestra.run.smithi110.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 93 kB/s | 61 kB 00:00 2024-09-11T22:02:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:02:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:02:06.893 INFO:teuthology.orchestra.run.smithi110.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-11T22:02:06.960 INFO:teuthology.orchestra.run.smithi110.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 288 kB/s | 245 kB 00:00 2024-09-11T22:02:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:02:07.033 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:02:07.064 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:02:07.077 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:02:07.112 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:02:07.126 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:02:07.156 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:02:07.423 INFO:teuthology.orchestra.run.smithi110.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 272 kB/s | 248 kB 00:00 2024-09-11T22:02:07.452 INFO:teuthology.orchestra.run.smithi110.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 39 kB/s | 19 kB 00:00 2024-09-11T22:02:07.486 INFO:teuthology.orchestra.run.smithi110.stdout:(80/156): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-09-11T22:02:08.076 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:02:08.078 INFO:teuthology.orchestra.run.smithi110.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-09-11T22:02:08.113 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:02:08.180 INFO:teuthology.orchestra.run.smithi110.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 228 kB/s | 172 kB 00:00 2024-09-11T22:02:08.208 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:02:08.215 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:02:08.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:02:08.388 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:02:08.623 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:02:08.648 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:02:08.648 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:08.648 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:02:08.648 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:02:08.648 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:02:08.648 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:02:08.711 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-11T22:02:08.810 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:02:08.871 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:02:08.970 INFO:teuthology.orchestra.run.smithi110.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 281 kB/s | 250 kB 00:00 2024-09-11T22:02:09.046 INFO:teuthology.orchestra.run.smithi110.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 256 kB/s | 222 kB 00:00 2024-09-11T22:02:09.213 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:02:09.241 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:02:09.338 INFO:teuthology.orchestra.run.smithi110.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 123 kB/s | 77 kB 00:00 2024-09-11T22:02:09.344 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:02:09.504 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:02:09.535 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:02:09.559 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:02:09.559 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:09.559 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:02:09.559 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:02:09.559 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:02:09.559 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:09.580 INFO:teuthology.orchestra.run.smithi110.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 69 kB/s | 42 kB 00:00 2024-09-11T22:02:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:02:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:02:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:02:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:02:09.817 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:02:09.852 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:02:09.875 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:02:09.876 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:09.876 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:02:09.876 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:02:09.876 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:02:09.876 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:09.880 INFO:teuthology.orchestra.run.smithi110.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 323 kB/s | 269 kB 00:00 2024-09-11T22:02:09.919 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:02:09.942 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:02:09.942 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:09.942 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:02:09.942 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:09.994 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:02:10.018 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:02:10.018 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:10.018 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:02:10.018 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:02:10.018 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:02:10.018 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:10.072 INFO:teuthology.orchestra.run.smithi110.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-11T22:02:10.320 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:02:10.344 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:02:10.344 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:10.344 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:02:10.344 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:02:10.344 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:02:10.344 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:10.447 INFO:teuthology.orchestra.run.smithi110.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 324 kB/s | 279 kB 00:00 2024-09-11T22:02:10.558 INFO:teuthology.orchestra.run.smithi110.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-09-11T22:02:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:02:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:02:10.739 INFO:teuthology.orchestra.run.smithi110.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 80 kB/s | 54 kB 00:00 2024-09-11T22:02:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:02:10.746 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:10.746 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:02:10.746 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:02:10.746 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:02:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:02:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:02:10.817 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:10.817 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:02:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:02:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:02:10.879 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:10.880 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:02:10.880 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:02:10.880 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:02:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:11.140 INFO:teuthology.orchestra.run.smithi110.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 69 kB/s | 42 kB 00:00 2024-09-11T22:02:11.190 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:02:11.214 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:02:11.214 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:11.214 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:02:11.214 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:02:11.214 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:02:11.214 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:11.331 INFO:teuthology.orchestra.run.smithi110.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 75 kB/s | 44 kB 00:00 2024-09-11T22:02:11.923 INFO:teuthology.orchestra.run.smithi110.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T22:02:11.949 INFO:teuthology.orchestra.run.smithi110.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 58 kB/s | 36 kB 00:00 2024-09-11T22:02:11.974 INFO:teuthology.orchestra.run.smithi110.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 265 kB/s | 221 kB 00:00 2024-09-11T22:02:12.556 INFO:teuthology.orchestra.run.smithi110.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-09-11T22:02:12.633 INFO:teuthology.orchestra.run.smithi110.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-09-11T22:02:12.833 INFO:teuthology.orchestra.run.smithi110.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 344 kB/s | 304 kB 00:00 2024-09-11T22:02:13.225 INFO:teuthology.orchestra.run.smithi110.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 143 kB/s | 96 kB 00:00 2024-09-11T22:02:13.325 INFO:teuthology.orchestra.run.smithi110.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-09-11T22:02:13.501 INFO:teuthology.orchestra.run.smithi110.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2024-09-11T22:02:13.526 INFO:teuthology.orchestra.run.smithi110.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-09-11T22:02:13.768 INFO:teuthology.orchestra.run.smithi110.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-09-11T22:02:13.793 INFO:teuthology.orchestra.run.smithi110.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-11T22:02:13.818 INFO:teuthology.orchestra.run.smithi110.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-09-11T22:02:13.852 INFO:teuthology.orchestra.run.smithi110.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 776 kB/s | 26 kB 00:00 2024-09-11T22:02:13.877 INFO:teuthology.orchestra.run.smithi110.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T22:02:13.903 INFO:teuthology.orchestra.run.smithi110.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T22:02:13.936 INFO:teuthology.orchestra.run.smithi110.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-11T22:02:13.995 INFO:teuthology.orchestra.run.smithi110.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-11T22:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-11T22:02:14.121 INFO:teuthology.orchestra.run.smithi110.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-09-11T22:02:14.146 INFO:teuthology.orchestra.run.smithi110.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-09-11T22:02:14.171 INFO:teuthology.orchestra.run.smithi110.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-11T22:02:14.197 INFO:teuthology.orchestra.run.smithi110.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-11T22:02:14.226 INFO:teuthology.orchestra.run.smithi110.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.1 MB/s | 172 kB 00:00 2024-09-11T22:02:14.259 INFO:teuthology.orchestra.run.smithi110.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.6 MB/s | 358 kB 00:00 2024-09-11T22:02:14.293 INFO:teuthology.orchestra.run.smithi110.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-11T22:02:14.318 INFO:teuthology.orchestra.run.smithi110.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T22:02:14.352 INFO:teuthology.orchestra.run.smithi110.stdout:(122/156): protobuf-compiler-3.14.0-13.el9.x86_ 841 kB/s | 863 kB 00:01 2024-09-11T22:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-11T22:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 766 kB/s | 18 kB 00:00 2024-09-11T22:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-09-11T22:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-11T22:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 330 kB/s | 19 kB 00:00 2024-09-11T22:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 334 kB/s | 20 kB 00:00 2024-09-11T22:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T22:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T22:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-11T22:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-11T22:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-11T22:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-11T22:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-11T22:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout:(137/156): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-09-11T22:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T22:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-11T22:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 850 kB/s | 36 kB 00:00 2024-09-11T22:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-11T22:02:14.907 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-11T22:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-11T22:02:14.948 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-11T22:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-11T22:02:14.989 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T22:02:15.021 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-11T22:02:15.057 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T22:02:15.254 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:02:15.286 INFO:teuthology.orchestra.run.smithi110.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-11T22:02:15.295 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:02:15.325 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:02:15.344 INFO:teuthology.orchestra.run.smithi110.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 9.3 MB/s | 2.3 MB 00:00 2024-09-11T22:02:15.355 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:02:15.393 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:02:15.403 INFO:teuthology.orchestra.run.smithi110.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 8.7 MB/s | 3.4 MB 00:00 2024-09-11T22:02:15.428 INFO:teuthology.orchestra.run.smithi110.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 488 kB/s | 41 kB 00:00 2024-09-11T22:02:15.470 INFO:teuthology.orchestra.run.smithi110.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 9.8 MB/s | 1.8 MB 00:00 2024-09-11T22:02:15.529 INFO:teuthology.orchestra.run.smithi110.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.9 MB/s | 249 kB 00:00 2024-09-11T22:02:15.571 INFO:teuthology.orchestra.run.smithi110.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-11T22:02:15.621 INFO:teuthology.orchestra.run.smithi110.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-11T22:02:15.627 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:02:15.627 INFO:teuthology.orchestra.run.smithi110.stdout:Total 10 MB/s | 226 MB 00:21 2024-09-11T22:02:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:02:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:02:15.884 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:02:15.916 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:02:15.943 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:02:15.999 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:02:16.040 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:02:16.061 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:02:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:02:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:02:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:02:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:02:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:02:16.800 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-11T22:02:16.859 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:02:16.901 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:02:16.912 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:02:16.922 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:02:16.951 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-11T22:02:16.951 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-11T22:02:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:02:17.048 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:02:17.148 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:02:17.181 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:02:17.206 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:02:17.233 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:02:17.233 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:02:17.316 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:02:17.316 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:02:17.392 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:02:17.417 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:02:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:02:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:02:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:02:17.603 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:02:17.694 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:02:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:02:17.759 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:02:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:02:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:02:17.971 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:02:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:02:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:02:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:02:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:02:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:02:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:02:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:02:18.237 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:02:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:02:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:02:18.422 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:02:18.422 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:02:18.506 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:02:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:02:19.323 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-11T22:02:19.323 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-11T22:02:23.391 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:02:23.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:02:23.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:02:23.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:02:23.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:02:23.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:02:23.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:02:23.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:02:23.405 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:02:23.405 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:02:23.405 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:02:23.405 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:02:23.406 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:02:23.407 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:02:23.408 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:02:23.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:02:23.410 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:02:23.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:02:23.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:02:23.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:02:23.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:02:23.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:02:23.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:02:24.294 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:02:24.294 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:02:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:02:24.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:02:24.308 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:02:24.308 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:02:24.308 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:02:24.309 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:02:24.310 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:02:24.311 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:02:24.312 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:02:24.313 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:02:24.314 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:02:24.315 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:02:24.316 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:02:24.317 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:02:24.318 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:02:25.609 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-11T22:02:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:02:25.771 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:02:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:02:25.868 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:02:25.893 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:02:25.946 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:02:26.116 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.117 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.118 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:02:26.119 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:02:26.120 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:02:26.121 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:02:26.122 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.123 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:02:26.124 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:02:26.125 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:02:26.126 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:02:26.127 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:02:26.128 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:02:26.129 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:02:26.130 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:02:26.131 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:02:26.131 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:02:26.131 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-09-11T22:02:26.339 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:02:26.428 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:02:26.550 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:02:26.567 DEBUG:teuthology.parallel:result is None 2024-09-11T22:02:26.579 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:02:26.631 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:02:26.631 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.634 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:02:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:02:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:02:26.637 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:02:26.638 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:02:26.639 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:02:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:02:26.641 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:02:26.642 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:02:26.643 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.644 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:02:26.645 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:02:26.646 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:02:26.646 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T22:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:02:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:02:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:02:26.941 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:02:27.030 DEBUG:teuthology.parallel:result is None 2024-09-11T22:02:27.046 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:02:27.076 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:02:27.122 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:02:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:02:27.544 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:02:27.667 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:02:27.779 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:02:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:02:28.206 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:02:28.495 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:02:28.593 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:02:28.630 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:02:28.667 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:02:28.731 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:02:28.776 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:02:28.809 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:02:28.860 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:02:28.908 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:02:28.937 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:02:29.005 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:02:29.059 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:02:29.091 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:02:29.135 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:02:29.210 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:02:29.245 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:02:29.297 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:02:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:02:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:02:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:02:29.530 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:02:29.663 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:02:29.709 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:02:30.230 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:02:30.313 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:02:30.343 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:02:30.385 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:02:30.432 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:02:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:02:30.639 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:02:30.693 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:02:30.748 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:02:30.821 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:02:30.853 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:02:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:02:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:02:31.006 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:02:31.048 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:02:31.094 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:02:31.237 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:02:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:02:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:02:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:02:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:02:31.561 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:02:31.653 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:02:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:02:31.819 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:02:31.858 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:02:31.897 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:02:31.937 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:02:31.968 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:02:31.998 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:02:32.033 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:02:32.066 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:02:32.108 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:02:32.142 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:02:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:02:32.281 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:02:32.318 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:02:32.421 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:02:32.477 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:02:33.198 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:02:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:02:33.275 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:02:33.373 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:02:33.422 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:02:33.475 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:02:33.562 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:02:33.594 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:02:33.778 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:02:33.878 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:02:33.891 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:02:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:02:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:02:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:02:35.964 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:02:36.000 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:02:36.042 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:02:36.350 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:02:36.405 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:02:36.479 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:02:36.509 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:02:36.545 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:02:37.023 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:02:37.050 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:02:37.121 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:02:37.143 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:02:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:02:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:02:40.027 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:02:40.068 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:02:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:02:40.160 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:02:40.160 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:02:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:02:52.134 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:02:52.134 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-09-11T22:02:52.135 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-09-11T22:02:52.135 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-09-11T22:02:52.135 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-09-11T22:02:52.135 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-09-11T22:02:52.135 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-09-11T22:02:52.136 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-09-11T22:02:52.136 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:02:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:02:52.324 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:52.325 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:02:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:53.821 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:02:53.847 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:02:53.847 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:53.847 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:02:53.847 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:02:53.847 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:02:53.848 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:02:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:02:54.103 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:54.103 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:02:54.103 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:02:54.104 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:02:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:54.530 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:02:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:02:54.553 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:54.553 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:02:54.553 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:02:54.553 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:02:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:02:54.709 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:02:54.764 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:02:54.958 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:02:55.006 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:02:55.076 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:02:55.164 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:02:55.311 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:02:55.366 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:02:55.397 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:02:55.425 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:02:55.884 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:02:55.924 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:02:55.964 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:02:56.026 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:02:56.064 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:02:57.099 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:02:57.133 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:02:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:02:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:02:58.217 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:02:58.395 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:02:58.427 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:02:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:02:58.450 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:58.450 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:02:58.450 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:02:58.450 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:02:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:02:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:02:58.611 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:02:58.679 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:02:59.596 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:02:59.631 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:02:59.654 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:02:59.654 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:59.654 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:02:59.654 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:02:59.654 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:02:59.654 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:59.698 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:02:59.720 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:02:59.721 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:59.721 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:02:59.721 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:02:59.788 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:02:59.788 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:02:59.788 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:02:59.788 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:02:59.788 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:02:59.788 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:03:00.091 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:03:00.114 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:03:00.114 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:03:00.114 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:03:00.115 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:03:00.115 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:03:00.115 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:03:04.679 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:03:04.729 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:03:04.769 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:03:04.802 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:03:04.830 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:03:05.023 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:03:05.076 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:03:05.819 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:03:05.856 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:03:05.876 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:03:06.695 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:03:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:03:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:03:06.930 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:03:06.965 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:03:06.990 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:03:07.033 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:03:07.033 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:03:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:03:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:03:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:03:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:03:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:03:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:03:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:03:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:03:07.504 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:03:07.569 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:03:13.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:03:13.167 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:03:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:03:15.705 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:03:15.705 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:03:15.705 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:03:15.706 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.707 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:03:15.708 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:03:15.709 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:03:15.710 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:03:15.711 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:03:15.712 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:03:15.713 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:03:15.714 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:03:15.715 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:03:15.716 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:03:15.717 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:03:15.717 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:03:15.717 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:03:15.717 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:03:15.717 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:03:15.717 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-11T22:03:16.146 DEBUG:teuthology.parallel:result is None 2024-09-11T22:03:16.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:03:16.296 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:03:16.323 INFO:teuthology.orchestra.run.smithi092.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:03:16.324 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:03:16.325 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:03:16.327 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:03:16.466 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:03:16.490 INFO:teuthology.orchestra.run.smithi110.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:03:16.490 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:03:16.490 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:03:16.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:03:16.628 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:03:16.655 INFO:teuthology.orchestra.run.smithi153.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:03:16.655 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:03:16.655 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:03:16.657 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T22:03:16.657 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:03:16.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:03:16.691 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:03:16.691 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:03:16.718 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:03:16.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:03:16.751 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T22:03:16.752 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:03:16.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:03:16.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:03:16.852 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:03:16.852 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:03:16.877 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:03:16.942 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:03:16.942 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:03:16.971 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:03:17.041 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T22:03:17.042 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:03:17.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:03:17.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:03:17.141 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:03:17.141 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:03:17.165 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:03:17.230 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:03:17.230 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:03:17.255 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:03:17.323 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T22:03:17.323 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:03:17.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:03:17.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:03:17.413 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:03:17.413 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:03:17.438 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:03:17.503 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:03:17.503 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:03:17.539 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:03:17.604 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T22:03:17.699 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-11T22:03:17.699 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:03:17.700 INFO:tasks.cephadm:Cluster fsid is a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:03:17.700 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T22:03:17.700 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.92', 'mon.b': '172.21.15.110', 'mon.c': '172.21.15.153'} 2024-09-11T22:03:17.701 INFO:tasks.cephadm:First mon is mon.a on smithi092 2024-09-11T22:03:17.701 INFO:tasks.cephadm:First mgr is x 2024-09-11T22:03:17.701 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T22:03:17.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-09-11T22:03:17.730 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-09-11T22:03:17.755 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-09-11T22:03:17.780 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T22:03:17.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:03:17.918 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T22:03:18.052 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:03:18.054 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T22:03:18.054 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T22:03:18.054 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:18.425 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:03 /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:18.425 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:18.787 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:03 /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:18.787 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:19.270 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:03 /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:19.270 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:19.290 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:19.311 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:03:19.346 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T22:03:19.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:03:19.349 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:03:19.354 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:03:20.976 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:03:21.013 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:03:21.043 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:03:53.041 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:03:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:03:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:03:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-09-11T22:03:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:03:53.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:03:53.041 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:04:06.255 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-11T22:04:06.255 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:04:06.255 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:04:06.255 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-09-11T22:04:06.256 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:04:06.256 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-09-11T22:04:06.256 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-11T22:04:11.806 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T22:04:11.806 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:04:11.806 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:04:11.806 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-09-11T22:04:11.807 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:04:11.807 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-11T22:04:11.807 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T22:04:11.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-09-11T22:04:11.865 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-09-11T22:04:11.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-09-11T22:04:11.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-09-11T22:04:11.953 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-09-11T22:04:11.977 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-09-11T22:04:12.007 INFO:tasks.cephadm:Writing seed config... 2024-09-11T22:04:12.008 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T22:04:12.008 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T22:04:12.008 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T22:04:12.008 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T22:04:12.008 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T22:04:12.009 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-11T22:04:12.010 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T22:04:12.011 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T22:04:12.012 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-11T22:04:12.012 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:04:12.012 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T22:04:12.029 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a6709e28-7089-11ef-bcea-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T22:04:12.029 DEBUG:teuthology.orchestra.run.smithi092:mon.a> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T22:04:12.072 DEBUG:teuthology.orchestra.run.smithi092:mgr.x> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T22:04:12.115 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T22:04:12.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid a6709e28-7089-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.92 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:04:12.320 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:04:12.320 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'a6709e28-7089-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.92', '--skip-admin-label'] 2024-09-11T22:04:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T22:04:12.320 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying podman|docker is present... 2024-09-11T22:04:12.344 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 5.2.2 2024-09-11T22:04:12.344 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying lvm2 is present... 2024-09-11T22:04:12.344 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying time synchronization is in place... 2024-09-11T22:04:12.351 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T22:04:12.352 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T22:04:12.358 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T22:04:12.358 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T22:04:12.365 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-09-11T22:04:12.371 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-09-11T22:04:12.371 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-09-11T22:04:12.371 INFO:teuthology.orchestra.run.smithi092.stdout:Repeating the final host check... 2024-09-11T22:04:12.420 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 5.2.2 2024-09-11T22:04:12.420 INFO:teuthology.orchestra.run.smithi092.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T22:04:12.420 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl is present 2024-09-11T22:04:12.421 INFO:teuthology.orchestra.run.smithi092.stdout:lvcreate is present 2024-09-11T22:04:12.427 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T22:04:12.427 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T22:04:12.433 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T22:04:12.433 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T22:04:12.439 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-09-11T22:04:12.446 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-09-11T22:04:12.446 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-09-11T22:04:12.446 INFO:teuthology.orchestra.run.smithi092.stdout:Host looks OK 2024-09-11T22:04:12.446 INFO:teuthology.orchestra.run.smithi092.stdout:Cluster fsid: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:12.446 INFO:teuthology.orchestra.run.smithi092.stdout:Acquiring lock 140567538082816 on /run/cephadm/a6709e28-7089-11ef-bcea-c7b262605968.lock 2024-09-11T22:04:12.446 INFO:teuthology.orchestra.run.smithi092.stdout:Lock 140567538082816 acquired on /run/cephadm/a6709e28-7089-11ef-bcea-c7b262605968.lock 2024-09-11T22:04:12.447 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 3300 ... 2024-09-11T22:04:12.447 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 6789 ... 2024-09-11T22:04:12.447 INFO:teuthology.orchestra.run.smithi092.stdout:Base mon IP(s) is [172.21.15.92:3300, 172.21.15.92:6789], mon addrv is [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-11T22:04:12.450 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.92 metric 100 2024-09-11T22:04:12.450 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.92 metric 100 2024-09-11T22:04:12.453 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T22:04:12.453 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T22:04:12.453 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T22:04:12.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T22:04:12.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T22:04:12.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T22:04:12.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T22:04:12.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-11T22:04:12.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T22:04:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-09-11T22:04:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-09-11T22:04:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T22:04:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T22:04:12.458 INFO:teuthology.orchestra.run.smithi092.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:04:12.931 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T22:04:12.931 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:04:12.931 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T22:04:12.931 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T22:04:12.932 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T22:04:12.932 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T22:04:12.932 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T22:04:13.852 INFO:teuthology.orchestra.run.smithi092.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T22:04:13.853 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T22:04:13.853 INFO:teuthology.orchestra.run.smithi092.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T22:04:14.768 INFO:teuthology.orchestra.run.smithi092.stdout:stat: stdout 167 167 2024-09-11T22:04:14.768 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial keys... 2024-09-11T22:04:15.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQDfE+Jm8SoeBRAANCG47OaLsR7U85EanvoQkw== 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQDgE+JmF33pARAArz9VcjL9tqZeMtJf/lhzZw== 2024-09-11T22:04:26.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQDmE+JmQBAQKhAAn2Jjk6VgP+6Oz+BF8qF5Bg== 2024-09-11T22:04:26.657 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial monmap... 2024-09-11T22:04:27.562 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T22:04:27.562 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T22:04:27.562 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:27.562 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T22:04:27.563 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool for a [v2:172.21.15.92:3300,v1:172.21.15.92:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T22:04:27.563 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = quincy 2024-09-11T22:04:27.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: set fsid to a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:27.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T22:04:27.563 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:04:27.563 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mon... 2024-09-11T22:04:28.502 INFO:teuthology.orchestra.run.smithi092.stdout:create mon.a on 2024-09-11T22:04:28.709 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T22:04:28.855 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T22:04:29.039 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a6709e28-7089-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a6709e28-7089-11ef-bcea-c7b262605968.target. 2024-09-11T22:04:29.039 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a6709e28-7089-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a6709e28-7089-11ef-bcea-c7b262605968.target. 2024-09-11T22:04:29.266 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a 2024-09-11T22:04:29.267 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T22:04:29.428 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a6709e28-7089-11ef-bcea-c7b262605968.target.wants/ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a6709e28-7089-11ef-bcea-c7b262605968@.service. 2024-09-11T22:04:29.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:29 smithi092 ceph-mon[30804]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:04:29.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:29 smithi092 bash[30785]: 449f04b5e883ef340f57f89d7becf4437e862eb77eaecc76f464fbd2500bf581 2024-09-11T22:04:29.910 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:04:29.910 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T22:04:29.917 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:04:29.917 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T22:04:29.917 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T22:04:29.918 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T22:04:29.918 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon to start... 2024-09-11T22:04:29.918 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon... 2024-09-11T22:04:30.162 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:29 smithi092 systemd[1]: Started Ceph mon.a for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:04:30.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:04:30.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: monmap epoch 1 2024-09-11T22:04:30.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:30.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: last_changed 2024-09-11T22:04:27.004735+0000 2024-09-11T22:04:30.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: created 2024-09-11T22:04:27.004735+0000 2024-09-11T22:04:30.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: min_mon_release 19 (squid) 2024-09-11T22:04:30.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: election_strategy: 1 2024-09-11T22:04:30.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-11T22:04:30.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: fsmap 2024-09-11T22:04:30.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T22:04:30.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: mgrmap e1: no daemons active 2024-09-11T22:04:30.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:30 smithi092 ceph-mon[30804]: from='client.? 172.21.15.92:0/173070358' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T22:04:30.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T22:04:30.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout id: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:30.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T22:04:30.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:30.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout services: 2024-09-11T22:04:30.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.525488s) 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout data: 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.324+0000 7f095fc33640 1 Processor -- start 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.325+0000 7f095fc33640 1 -- start start 2024-09-11T22:04:30.988 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.325+0000 7f095fc33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095810a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.325+0000 7f095fc33640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0958075840 con 0x7f0958107ed0 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.325+0000 7f095d9cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095810a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.325+0000 7f095d9cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095810a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43356/0 (socket says 172.21.15.92:43356) 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.325+0000 7f095d9cf640 1 -- 172.21.15.92:0/2961256645 learned_addr learned my addr 172.21.15.92:0/2961256645 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.325+0000 7f095d9cf640 1 -- 172.21.15.92:0/2961256645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095810a800 con 0x7f0958107ed0 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095d9cf640 1 --2- 172.21.15.92:0/2961256645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095810a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f094c009820 tx=0x7f094c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d96ee2f49a8b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095c9cd640 1 -- 172.21.15.92:0/2961256645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c037070 con 0x7f0958107ed0 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095c9cd640 1 -- 172.21.15.92:0/2961256645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c032440 con 0x7f0958107ed0 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095c9cd640 1 -- 172.21.15.92:0/2961256645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c032740 con 0x7f0958107ed0 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095fc33640 1 -- 172.21.15.92:0/2961256645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 msgr2=0x7f095810a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095fc33640 1 --2- 172.21.15.92:0/2961256645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095810a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f094c009820 tx=0x7f094c02a270 comp rx=0 tx=0).stop 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095fc33640 1 -- 172.21.15.92:0/2961256645 shutdown_connections 2024-09-11T22:04:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095fc33640 1 --2- 172.21.15.92:0/2961256645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095810a2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.328+0000 7f095fc33640 1 -- 172.21.15.92:0/2961256645 >> 172.21.15.92:0/2961256645 conn(0x7f0958101a90 msgr2=0x7f0958103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095fc33640 1 -- 172.21.15.92:0/2961256645 shutdown_connections 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095fc33640 1 -- 172.21.15.92:0/2961256645 wait complete. 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095fc33640 1 Processor -- start 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095fc33640 1 -- start start 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095fc33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095fc33640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f095810ace0 con 0x7f0958107ed0 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095d9cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095d9cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43366/0 (socket says 172.21.15.92:43366) 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.329+0000 7f095d9cf640 1 -- 172.21.15.92:0/173070358 learned_addr learned my addr 172.21.15.92:0/173070358 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f095d9cf640 1 -- 172.21.15.92:0/173070358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095819b110 con 0x7f0958107ed0 2024-09-11T22:04:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f095d9cf640 1 --2- 172.21.15.92:0/173070358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095819abd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f094c0090b0 tx=0x7f094c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f094affd640 1 -- 172.21.15.92:0/173070358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c041070 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f094affd640 1 -- 172.21.15.92:0/173070358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c03c7a0 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095819b3a0 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f094affd640 1 -- 172.21.15.92:0/173070358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c037040 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095819e090 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f094affd640 1 -- 172.21.15.92:0/173070358 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f094c044070 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.330+0000 7f094affd640 1 -- 172.21.15.92:0/173070358 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f094c03d3f0 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.331+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0958108330 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.333+0000 7f094affd640 1 -- 172.21.15.92:0/173070358 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f094c046450 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.392+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f095802d910 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.392+0000 7f094affd640 1 -- 172.21.15.92:0/173070358 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f094c046630 con 0x7f0958107ed0 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.393+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 msgr2=0x7f095819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:30.991 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.393+0000 7f095fc33640 1 --2- 172.21.15.92:0/173070358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095819abd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f094c0090b0 tx=0x7f094c0046c0 comp rx=0 tx=0).stop 2024-09-11T22:04:30.992 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.393+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 shutdown_connections 2024-09-11T22:04:30.992 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.393+0000 7f095fc33640 1 --2- 172.21.15.92:0/173070358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0958107ed0 0x7f095819abd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:30.992 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.393+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 >> 172.21.15.92:0/173070358 conn(0x7f0958101a90 msgr2=0x7f0958102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:30.992 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.393+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 shutdown_connections 2024-09-11T22:04:30.992 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:30.393+0000 7f095fc33640 1 -- 172.21.15.92:0/173070358 wait complete. 2024-09-11T22:04:30.992 INFO:teuthology.orchestra.run.smithi092.stdout:mon is available 2024-09-11T22:04:30.992 INFO:teuthology.orchestra.run.smithi092.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T22:04:32.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:31 smithi092 ceph-mon[30804]: from='client.? 172.21.15.92:0/1698908262' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T22:04:32.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:31 smithi092 ceph-mon[30804]: from='client.? 172.21.15.92:0/1698908262' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T22:04:33.413 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T22:04:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-11T22:04:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T22:04:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.381+0000 7fe1757ec640 1 Processor -- start 2024-09-11T22:04:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.381+0000 7fe1757ec640 1 -- start start 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.382+0000 7fe1757ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe170106110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.382+0000 7fe1757ec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe17007bdb0 con 0x7fe170105d10 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.382+0000 7fe16effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe170106110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.382+0000 7fe16effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe170106110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43380/0 (socket says 172.21.15.92:43380) 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.382+0000 7fe16effd640 1 -- 172.21.15.92:0/2830885502 learned_addr learned my addr 172.21.15.92:0/2830885502 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.382+0000 7fe16effd640 1 -- 172.21.15.92:0/2830885502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe170106650 con 0x7fe170105d10 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.383+0000 7fe16effd640 1 --2- 172.21.15.92:0/2830885502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe170106110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe160009820 tx=0x7fe16002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b6b5dcdbe3c181 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.383+0000 7fe16dffb640 1 -- 172.21.15.92:0/2830885502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160037070 con 0x7fe170105d10 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.383+0000 7fe16dffb640 1 -- 172.21.15.92:0/2830885502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe160032440 con 0x7fe170105d10 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.383+0000 7fe16dffb640 1 -- 172.21.15.92:0/2830885502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160032740 con 0x7fe170105d10 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 -- 172.21.15.92:0/2830885502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 msgr2=0x7fe170106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 --2- 172.21.15.92:0/2830885502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe170106110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe160009820 tx=0x7fe16002a270 comp rx=0 tx=0).stop 2024-09-11T22:04:33.421 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 -- 172.21.15.92:0/2830885502 shutdown_connections 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 --2- 172.21.15.92:0/2830885502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe170106110 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 -- 172.21.15.92:0/2830885502 >> 172.21.15.92:0/2830885502 conn(0x7fe170101ae0 msgr2=0x7fe170103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 -- 172.21.15.92:0/2830885502 shutdown_connections 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 -- 172.21.15.92:0/2830885502 wait complete. 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 Processor -- start 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.384+0000 7fe1757ec640 1 -- start start 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe1757ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe17019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe1757ec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe170106bf0 con 0x7fe170105d10 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe16effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe17019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe16effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe17019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43394/0 (socket says 172.21.15.92:43394) 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe16effd640 1 -- 172.21.15.92:0/1698908262 learned_addr learned my addr 172.21.15.92:0/1698908262 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe16effd640 1 -- 172.21.15.92:0/1698908262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17019d2f0 con 0x7fe170105d10 2024-09-11T22:04:33.422 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe16effd640 1 --2- 172.21.15.92:0/1698908262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe17019cdb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe1600090b0 tx=0x7fe1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe157fff640 1 -- 172.21.15.92:0/1698908262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160041070 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe157fff640 1 -- 172.21.15.92:0/1698908262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16003c7a0 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17019d580 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe157fff640 1 -- 172.21.15.92:0/1698908262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160037040 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a0270 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.385+0000 7fe157fff640 1 -- 172.21.15.92:0/1698908262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe160044070 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.386+0000 7fe157fff640 1 -- 172.21.15.92:0/1698908262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe16003d3f0 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.386+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170106110 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.388+0000 7fe157fff640 1 -- 172.21.15.92:0/1698908262 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe160046450 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.445+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe1701a1940 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.448+0000 7fe157fff640 1 -- 172.21.15.92:0/1698908262 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fe160046630 con 0x7fe170105d10 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.449+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 msgr2=0x7fe17019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.449+0000 7fe1757ec640 1 --2- 172.21.15.92:0/1698908262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe17019cdb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe1600090b0 tx=0x7fe1600046c0 comp rx=0 tx=0).stop 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.449+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 shutdown_connections 2024-09-11T22:04:33.423 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.449+0000 7fe1757ec640 1 --2- 172.21.15.92:0/1698908262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe170105d10 0x7fe17019cdb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:33.424 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.449+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 >> 172.21.15.92:0/1698908262 conn(0x7fe170101ae0 msgr2=0x7fe170193390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:33.424 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.449+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 shutdown_connections 2024-09-11T22:04:33.424 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:31.449+0000 7fe1757ec640 1 -- 172.21.15.92:0/1698908262 wait complete. 2024-09-11T22:04:33.424 INFO:teuthology.orchestra.run.smithi092.stdout:Generating new minimal ceph.conf... 2024-09-11T22:04:34.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:33 smithi092 ceph-mon[30804]: from='client.? 172.21.15.92:0/3456815589' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:04:34.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.807+0000 7f53db19c640 1 Processor -- start 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.807+0000 7f53db19c640 1 -- start start 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.807+0000 7f53db19c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d4106720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.807+0000 7f53db19c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4106cf0 con 0x7f53d4106320 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.808+0000 7f53d8f38640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d4106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.808+0000 7f53d8f38640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d4106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43400/0 (socket says 172.21.15.92:43400) 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.808+0000 7f53d8f38640 1 -- 172.21.15.92:0/2597064800 learned_addr learned my addr 172.21.15.92:0/2597064800 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.808+0000 7f53d8f38640 1 -- 172.21.15.92:0/2597064800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4107520 con 0x7f53d4106320 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.809+0000 7f53d8f38640 1 --2- 172.21.15.92:0/2597064800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d4106720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009820 tx=0x7f53c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadd3999f86fe581 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.809+0000 7f53d37fe640 1 -- 172.21.15.92:0/2597064800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c4037070 con 0x7f53d4106320 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.809+0000 7f53d37fe640 1 -- 172.21.15.92:0/2597064800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f53c40328d0 con 0x7f53d4106320 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.809+0000 7f53d37fe640 1 -- 172.21.15.92:0/2597064800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c40307d0 con 0x7f53d4106320 2024-09-11T22:04:34.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.809+0000 7f53db19c640 1 -- 172.21.15.92:0/2597064800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 msgr2=0x7f53d4106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.809+0000 7f53db19c640 1 --2- 172.21.15.92:0/2597064800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d4106720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009820 tx=0x7f53c402a270 comp rx=0 tx=0).stop 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 -- 172.21.15.92:0/2597064800 shutdown_connections 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 --2- 172.21.15.92:0/2597064800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d4106720 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 -- 172.21.15.92:0/2597064800 >> 172.21.15.92:0/2597064800 conn(0x7f53d4101a90 msgr2=0x7f53d4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 -- 172.21.15.92:0/2597064800 shutdown_connections 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 -- 172.21.15.92:0/2597064800 wait complete. 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 Processor -- start 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 -- start start 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53db19c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410a8d0 con 0x7f53d4106320 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53d8f38640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.810+0000 7f53d8f38640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43416/0 (socket says 172.21.15.92:43416) 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53d8f38640 1 -- 172.21.15.92:0/3456815589 learned_addr learned my addr 172.21.15.92:0/3456815589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53d8f38640 1 -- 172.21.15.92:0/3456815589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d419a7d0 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53d8f38640 1 --2- 172.21.15.92:0/3456815589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d419a290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53c40097f0 tx=0x7f53c4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53d1ffb640 1 -- 172.21.15.92:0/3456815589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c4037040 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53d1ffb640 1 -- 172.21.15.92:0/3456815589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f53c4032bb0 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d419aa60 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53d1ffb640 1 -- 172.21.15.92:0/3456815589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c403a390 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d419d750 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.811+0000 7f53d1ffb640 1 -- 172.21.15.92:0/3456815589 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53c403a580 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.812+0000 7f53d1ffb640 1 -- 172.21.15.92:0/3456815589 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53c4044b50 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.812+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d410aa50 con 0x7f53d4106320 2024-09-11T22:04:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.814+0000 7f53d1ffb640 1 -- 172.21.15.92:0/3456815589 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f53c4032d60 con 0x7f53d4106320 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.872+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f53d419dc30 con 0x7f53d4106320 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.872+0000 7f53d1ffb640 1 -- 172.21.15.92:0/3456815589 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f53c403aa00 con 0x7f53d4106320 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.873+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 msgr2=0x7f53d419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.873+0000 7f53db19c640 1 --2- 172.21.15.92:0/3456815589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d419a290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53c40097f0 tx=0x7f53c4030d00 comp rx=0 tx=0).stop 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.873+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 shutdown_connections 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.873+0000 7f53db19c640 1 --2- 172.21.15.92:0/3456815589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d4106320 0x7f53d419a290 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.873+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 >> 172.21.15.92:0/3456815589 conn(0x7f53d4101a90 msgr2=0x7f53d41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.873+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 shutdown_connections 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:33.873+0000 7f53db19c640 1 -- 172.21.15.92:0/3456815589 wait complete. 2024-09-11T22:04:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:Restarting the monitor... 2024-09-11T22:04:34.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:34 smithi092 systemd[1]: Stopping Ceph mon.a for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:04:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:34 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a[30800]: 2024-09-11T22:04:34.677+0000 7f48dc684640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:04:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:34 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a[30800]: 2024-09-11T22:04:34.677+0000 7f48dc684640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T22:04:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:34 smithi092 podman[31048]: 2024-09-11 22:04:34.841499429 +0000 UTC m=+0.283606090 container died 449f04b5e883ef340f57f89d7becf4437e862eb77eaecc76f464fbd2500bf581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:04:35.429 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 podman[31048]: 2024-09-11 22:04:35.202928204 +0000 UTC m=+0.645034867 container cleanup 449f04b5e883ef340f57f89d7becf4437e862eb77eaecc76f464fbd2500bf581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-11T22:04:35.429 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 bash[31048]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a 2024-09-11T22:04:35.763 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 podman[31060]: 2024-09-11 22:04:35.428984432 +0000 UTC m=+0.580760963 container remove 449f04b5e883ef340f57f89d7becf4437e862eb77eaecc76f464fbd2500bf581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T22:04:35.763 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 systemd[1]: ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T22:04:35.763 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 systemd[1]: Stopped Ceph mon.a for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:04:35.763 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 systemd[1]: Starting Ceph mon.a for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:04:36.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 podman[31154]: 2024-09-11 22:04:35.763234531 +0000 UTC m=+0.098842444 container create 966a88dcdfc4fefe75aba5011c9d75ff907f83b4958aff94f4da125782d31922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T22:04:36.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 podman[31154]: 2024-09-11 22:04:35.680398052 +0000 UTC m=+0.016005973 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:04:36.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 podman[31154]: 2024-09-11 22:04:35.990835315 +0000 UTC m=+0.326443229 container init 966a88dcdfc4fefe75aba5011c9d75ff907f83b4958aff94f4da125782d31922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:04:36.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:35 smithi092 podman[31154]: 2024-09-11 22:04:35.993941622 +0000 UTC m=+0.329549537 container start 966a88dcdfc4fefe75aba5011c9d75ff907f83b4958aff94f4da125782d31922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T22:04:36.095 INFO:teuthology.orchestra.run.smithi092.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: pidfile_write: ignore empty --pid-file 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: load: jerasure load: lrc 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: RocksDB version: 7.9.2 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Git sha 0 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: DB SUMMARY 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: DB Session ID: 0Z2DVAVNGKQWP38M9NW0 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: CURRENT file: CURRENT 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T22:04:36.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104254 ; 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.error_if_exists: 0 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.create_if_missing: 0 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.paranoid_checks: 1 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.env: 0x5653e360f940 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.info_log: 0x5653e5b7b6e0 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T22:04:36.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.statistics: (nil) 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.use_fsync: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_log_file_size: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.allow_fallocate: 1 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.use_direct_reads: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.db_log_dir: 2024-09-11T22:04:36.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.wal_dir: 2024-09-11T22:04:36.272 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T22:04:36.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T22:04:36.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T22:04:36.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T22:04:36.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T22:04:36.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T22:04:36.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T22:04:36.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.write_buffer_manager: 0x5653e5b8adc0 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.unordered_write: 0 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T22:04:36.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.row_cache: None 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.wal_filter: None 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.two_write_queues: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.wal_compression: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.atomic_flush: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.log_readahead_size: 0 2024-09-11T22:04:36.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_background_jobs: 2 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_background_compactions: -1 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_subcompactions: 1 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T22:04:36.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_open_files: -1 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_background_flushes: -1 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Compression algorithms supported: 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kZSTD supported: 0 2024-09-11T22:04:36.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kXpressCompression supported: 0 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kBZip2Compression supported: 0 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kLZ4Compression supported: 1 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kZlibCompression supported: 1 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: kSnappyCompression supported: 1 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.merge_operator: 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_filter: None 2024-09-11T22:04:36.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5653e5b7b320) 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_type: 0 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_index_type: 0 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_shortening: 1 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: checksum: 4 2024-09-11T22:04:36.279 INFO:journalctl@ceph.mon.a.smithi092.stdout: no_block_cache: 0 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache: 0x5653e5b6f1f0 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_options: 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: capacity : 536870912 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: num_shard_bits : 4 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: strict_capacity_limit : 0 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_compressed: (nil) 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: persistent_cache: (nil) 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size: 4096 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size_deviation: 10 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_restart_interval: 16 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_block_restart_interval: 1 2024-09-11T22:04:36.280 INFO:journalctl@ceph.mon.a.smithi092.stdout: metadata_block_size: 4096 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: partition_filters: 0 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: use_delta_encoding: 1 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: filter_policy: bloomfilter 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: whole_key_filtering: 1 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: verify_compression: 0 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: format_version: 5 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_index_compression: 1 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_align: 0 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: max_auto_readahead_size: 262144 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: prepopulate_block_cache: 0 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: initial_auto_readahead_size: 8192 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T22:04:36.281 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression: NoCompression 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.num_levels: 7 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T22:04:36.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T22:04:36.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T22:04:36.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T22:04:36.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.inplace_update_support: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.bloom_locality: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.max_successive_merges: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T22:04:36.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.ttl: 2592000 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.enable_blob_files: false 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.min_blob_size: 0 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T22:04:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9581a4c8-9f10-4ccc-bc80-aef57f28b0e9 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726092276020458, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726092276021389, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97384, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14044, "raw_average_key_size": 49, "raw_value_size": 89705, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726092276, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9581a4c8-9f10-4ccc-bc80-aef57f28b0e9", "db_session_id": "0Z2DVAVNGKQWP38M9NW0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726092276021483, "job": 1, "event": "recovery_finished"} 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5653e5b9ce00 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: DB pointer 0x5653e5c98000 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T22:04:36.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** DB Stats ** 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: L0 2/0 99.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:04:36.289 INFO:journalctl@ceph.mon.a.smithi092.stdout: Sum 2/0 99.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 163.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T22:04:36.290 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: Block cache BinnedLRUCache@0x5653e5b6f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: starting mon.a rank 0 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???) e1 preinit fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???).mds e1 new map 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???).mds e1 print_map 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: e1 2024-09-11T22:04:36.291 INFO:journalctl@ceph.mon.a.smithi092.stdout: btime 2024-09-11T22:04:29:869139+0000 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout: legacy client fscid: -1 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout: No filesystems configured 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: monmap epoch 1 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:36.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: last_changed 2024-09-11T22:04:27.004735+0000 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: created 2024-09-11T22:04:27.004735+0000 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: min_mon_release 19 (squid) 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: election_strategy: 1 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: fsmap 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 ceph-mon[31173]: mgrmap e1: no daemons active 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 bash[31154]: 966a88dcdfc4fefe75aba5011c9d75ff907f83b4958aff94f4da125782d31922 2024-09-11T22:04:36.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:36 smithi092 systemd[1]: Started Ceph mon.a for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:04:37.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.615+0000 7f0655011640 1 Processor -- start 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.616+0000 7f0655011640 1 -- start start 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.616+0000 7f0655011640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f0650106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.616+0000 7f0655011640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0650106fe0 con 0x7f0650106610 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.616+0000 7f064ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f0650106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.616+0000 7f064ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f0650106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41698/0 (socket says 172.21.15.92:41698) 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.616+0000 7f064ed9d640 1 -- 172.21.15.92:0/1499333504 learned_addr learned my addr 172.21.15.92:0/1499333504 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.617+0000 7f064ed9d640 1 -- 172.21.15.92:0/1499333504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0650107160 con 0x7f0650106610 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.618+0000 7f064ed9d640 1 --2- 172.21.15.92:0/1499333504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f0650106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0640009820 tx=0x7f064002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22164988c212c138 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.618+0000 7f064dd9b640 1 -- 172.21.15.92:0/1499333504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0640037070 con 0x7f0650106610 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.618+0000 7f064dd9b640 1 -- 172.21.15.92:0/1499333504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f06400328d0 con 0x7f0650106610 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.618+0000 7f064dd9b640 1 -- 172.21.15.92:0/1499333504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06400307d0 con 0x7f0650106610 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.619+0000 7f0655011640 1 -- 172.21.15.92:0/1499333504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 msgr2=0x7f0650106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:37.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.619+0000 7f0655011640 1 --2- 172.21.15.92:0/1499333504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f0650106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0640009820 tx=0x7f064002a270 comp rx=0 tx=0).stop 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.619+0000 7f0655011640 1 -- 172.21.15.92:0/1499333504 shutdown_connections 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.619+0000 7f0655011640 1 --2- 172.21.15.92:0/1499333504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f0650106a10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.619+0000 7f0655011640 1 -- 172.21.15.92:0/1499333504 >> 172.21.15.92:0/1499333504 conn(0x7f0650100dc0 msgr2=0x7f0650103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.619+0000 7f0655011640 1 -- 172.21.15.92:0/1499333504 shutdown_connections 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.619+0000 7f0655011640 1 -- 172.21.15.92:0/1499333504 wait complete. 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.620+0000 7f0655011640 1 Processor -- start 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.620+0000 7f0655011640 1 -- start start 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.620+0000 7f0655011640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f065019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.620+0000 7f0655011640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0650105fb0 con 0x7f0650106610 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.620+0000 7f064ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f065019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.620+0000 7f064ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f065019d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41702/0 (socket says 172.21.15.92:41702) 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.620+0000 7f064ed9d640 1 -- 172.21.15.92:0/1266374397 learned_addr learned my addr 172.21.15.92:0/1266374397 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:37.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f064ed9d640 1 -- 172.21.15.92:0/1266374397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065019da80 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f064ed9d640 1 --2- 172.21.15.92:0/1266374397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f065019d540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0640032e10 tx=0x7f0640032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f0647fff640 1 -- 172.21.15.92:0/1266374397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0640042070 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f0647fff640 1 -- 172.21.15.92:0/1266374397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0640030c80 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065019dd10 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f0647fff640 1 -- 172.21.15.92:0/1266374397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0640037040 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06501a0a00 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f0647fff640 1 -- 172.21.15.92:0/1266374397 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0640045070 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.621+0000 7f0647fff640 1 -- 172.21.15.92:0/1266374397 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f06400479f0 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.622+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0650106a90 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.624+0000 7f0647fff640 1 -- 172.21.15.92:0/1266374397 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f064003c5f0 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.682+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f0650196f70 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.683+0000 7f0647fff640 1 -- 172.21.15.92:0/1266374397 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f064003c7d0 con 0x7f0650106610 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.684+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 msgr2=0x7f065019d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:37.250 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.684+0000 7f0655011640 1 --2- 172.21.15.92:0/1266374397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f065019d540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0640032e10 tx=0x7f0640032e40 comp rx=0 tx=0).stop 2024-09-11T22:04:37.251 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.684+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 shutdown_connections 2024-09-11T22:04:37.251 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.684+0000 7f0655011640 1 --2- 172.21.15.92:0/1266374397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0650106610 0x7f065019d540 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:37.251 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.684+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 >> 172.21.15.92:0/1266374397 conn(0x7f0650100dc0 msgr2=0x7f0650194140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:37.251 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.684+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 shutdown_connections 2024-09-11T22:04:37.251 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:36.684+0000 7f0655011640 1 -- 172.21.15.92:0/1266374397 wait complete. 2024-09-11T22:04:37.251 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T22:04:37.274 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:04:37.274 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mgr... 2024-09-11T22:04:37.274 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T22:04:37.275 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T22:04:37.533 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.x 2024-09-11T22:04:37.534 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T22:04:37.683 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a6709e28-7089-11ef-bcea-c7b262605968.target.wants/ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a6709e28-7089-11ef-bcea-c7b262605968@.service. 2024-09-11T22:04:37.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:37 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1266374397' entity='client.admin' 2024-09-11T22:04:38.201 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:04:38.201 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T22:04:38.209 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:04:38.210 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T22:04:38.210 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T22:04:38.210 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T22:04:38.219 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:04:38.220 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T22:04:38.228 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:04:38.228 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T22:04:38.228 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T22:04:38.228 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T22:04:38.228 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr to start... 2024-09-11T22:04:38.228 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr... 2024-09-11T22:04:38.352 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:38 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:38.273+0000 7f6e7c89f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:04:38.634 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:38 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:38.421+0000 7f6e7c89f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:04:38.634 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:38 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:38.487+0000 7f6e7c89f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:04:38.635 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:38 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:38.633+0000 7f6e7c89f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:04:38.887 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:38 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:38.777+0000 7f6e7c89f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:04:39.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:39 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2148631179' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:04:39.171 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:39 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:39.108+0000 7f6e7c89f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:04:39.171 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:39 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:39.169+0000 7f6e7c89f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:04:39.518 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:39 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:39.358+0000 7f6e7c89f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:04:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:04:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:04:29:869139+0000", 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T22:04:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:04:29.869766+0000", 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:04:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.960+0000 7faccb07c640 1 Processor -- start 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.960+0000 7faccb07c640 1 -- start start 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.961+0000 7faccb07c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4106720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.961+0000 7faccb07c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7facc4106cf0 con 0x7facc4106320 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.961+0000 7facc8e18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.961+0000 7facc8e18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41724/0 (socket says 172.21.15.92:41724) 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.961+0000 7facc8e18640 1 -- 172.21.15.92:0/3578103785 learned_addr learned my addr 172.21.15.92:0/3578103785 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.961+0000 7facc8e18640 1 -- 172.21.15.92:0/3578103785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc4107520 con 0x7facc4106320 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.962+0000 7facc8e18640 1 --2- 172.21.15.92:0/3578103785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4106720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7facb8009820 tx=0x7facb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d2ed77a87f835a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.962+0000 7facc37fe640 1 -- 172.21.15.92:0/3578103785 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facb8037070 con 0x7facc4106320 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.962+0000 7facc37fe640 1 -- 172.21.15.92:0/3578103785 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7facb80328d0 con 0x7facc4106320 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.962+0000 7facc37fe640 1 -- 172.21.15.92:0/3578103785 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facb80307d0 con 0x7facc4106320 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.963+0000 7faccb07c640 1 -- 172.21.15.92:0/3578103785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 msgr2=0x7facc4106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:39.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.963+0000 7faccb07c640 1 --2- 172.21.15.92:0/3578103785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4106720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7facb8009820 tx=0x7facb802a270 comp rx=0 tx=0).stop 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.963+0000 7faccb07c640 1 -- 172.21.15.92:0/3578103785 shutdown_connections 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.963+0000 7faccb07c640 1 --2- 172.21.15.92:0/3578103785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4106720 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.963+0000 7faccb07c640 1 -- 172.21.15.92:0/3578103785 >> 172.21.15.92:0/3578103785 conn(0x7facc4101a90 msgr2=0x7facc4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.963+0000 7faccb07c640 1 -- 172.21.15.92:0/3578103785 shutdown_connections 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.963+0000 7faccb07c640 1 -- 172.21.15.92:0/3578103785 wait complete. 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7faccb07c640 1 Processor -- start 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7faccb07c640 1 -- start start 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7faccb07c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7faccb07c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7facc410a8d0 con 0x7facc4106320 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7facc8e18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7facc8e18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4075420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41730/0 (socket says 172.21.15.92:41730) 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7facc8e18640 1 -- 172.21.15.92:0/2148631179 learned_addr learned my addr 172.21.15.92:0/2148631179 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7facc8e18640 1 -- 172.21.15.92:0/2148631179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc419e0e0 con 0x7facc4106320 2024-09-11T22:04:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7facc8e18640 1 --2- 172.21.15.92:0/2148631179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4075420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7facb8009940 tx=0x7facb8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.964+0000 7facc1ffb640 1 -- 172.21.15.92:0/2148631179 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facb8042070 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.965+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc419e2b0 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.965+0000 7facc1ffb640 1 -- 172.21.15.92:0/2148631179 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7facb8030c80 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.965+0000 7facc1ffb640 1 -- 172.21.15.92:0/2148631179 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facb8037040 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.965+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc419e650 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.965+0000 7facc1ffb640 1 -- 172.21.15.92:0/2148631179 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7facb8045070 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.965+0000 7facc1ffb640 1 -- 172.21.15.92:0/2148631179 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7facb80479e0 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.966+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc410aa50 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:38.969+0000 7facc1ffb640 1 -- 172.21.15.92:0/2148631179 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7facb803c5f0 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.030+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7facc419ea30 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.030+0000 7facc1ffb640 1 -- 172.21.15.92:0/2148631179 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7facb803c7d0 con 0x7facc4106320 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.031+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 msgr2=0x7facc4075420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.031+0000 7faccb07c640 1 --2- 172.21.15.92:0/2148631179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4075420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7facb8009940 tx=0x7facb8032e10 comp rx=0 tx=0).stop 2024-09-11T22:04:39.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.031+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 shutdown_connections 2024-09-11T22:04:39.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.031+0000 7faccb07c640 1 --2- 172.21.15.92:0/2148631179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4106320 0x7facc4075420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:39.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.031+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 >> 172.21.15.92:0/2148631179 conn(0x7facc4101a90 msgr2=0x7facc41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:39.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.031+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 shutdown_connections 2024-09-11T22:04:39.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:39.031+0000 7faccb07c640 1 -- 172.21.15.92:0/2148631179 wait complete. 2024-09-11T22:04:39.622 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (1/15)... 2024-09-11T22:04:40.518 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:40 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:40.241+0000 7f6e7c89f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:04:40.518 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:40 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:40.306+0000 7f6e7c89f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:04:41.272 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.018+0000 7f6e7c89f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:04:41.272 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.131+0000 7f6e7c89f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:04:41.545 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.271+0000 7f6e7c89f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:04:41.545 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.331+0000 7f6e7c89f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:04:41.545 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.477+0000 7f6e7c89f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:04:41.545 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.544+0000 7f6e7c89f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:04:41.812 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.603+0000 7f6e7c89f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:04:41.812 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.768+0000 7f6e7c89f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:04:42.067 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.836+0000 7f6e7c89f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:04:42.067 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.902+0000 7f6e7c89f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:04:42.067 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:41.989+0000 7f6e7c89f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:04:42.067 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:42 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:42.053+0000 7f6e7c89f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:04:42.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:42 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1617529969' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:04:42.518 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:42 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:42.117+0000 7f6e7c89f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:04:42.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:04:42.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:04:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:04:29:869139+0000", 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:04:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:04:29.869766+0000", 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f7aa9640 1 Processor -- start 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f7aa9640 1 -- start start 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f7aa9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f0106720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f7aa9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0106cf0 con 0x7f48f0106320 2024-09-11T22:04:42.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f5845640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f0106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f5845640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f0106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41732/0 (socket says 172.21.15.92:41732) 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f5845640 1 -- 172.21.15.92:0/4143530698 learned_addr learned my addr 172.21.15.92:0/4143530698 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.998+0000 7f48f5845640 1 -- 172.21.15.92:0/4143530698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f0107520 con 0x7f48f0106320 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.999+0000 7f48f5845640 1 --2- 172.21.15.92:0/4143530698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f0106720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009820 tx=0x7f48ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46bbb39e7f5e516 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.999+0000 7f48f4843640 1 -- 172.21.15.92:0/4143530698 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48ec037070 con 0x7f48f0106320 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.999+0000 7f48f4843640 1 -- 172.21.15.92:0/4143530698 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f48ec0328d0 con 0x7f48f0106320 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:41.999+0000 7f48f4843640 1 -- 172.21.15.92:0/4143530698 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48ec0307d0 con 0x7f48f0106320 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.000+0000 7f48f7aa9640 1 -- 172.21.15.92:0/4143530698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 msgr2=0x7f48f0106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.000+0000 7f48f7aa9640 1 --2- 172.21.15.92:0/4143530698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f0106720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009820 tx=0x7f48ec02a270 comp rx=0 tx=0).stop 2024-09-11T22:04:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.000+0000 7f48f7aa9640 1 -- 172.21.15.92:0/4143530698 shutdown_connections 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.000+0000 7f48f7aa9640 1 --2- 172.21.15.92:0/4143530698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f0106720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.000+0000 7f48f7aa9640 1 -- 172.21.15.92:0/4143530698 >> 172.21.15.92:0/4143530698 conn(0x7f48f0101a90 msgr2=0x7f48f0103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.000+0000 7f48f7aa9640 1 -- 172.21.15.92:0/4143530698 shutdown_connections 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.000+0000 7f48f7aa9640 1 -- 172.21.15.92:0/4143530698 wait complete. 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f7aa9640 1 Processor -- start 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f7aa9640 1 -- start start 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f7aa9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f7aa9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010a8d0 con 0x7f48f0106320 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f5845640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f5845640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41736/0 (socket says 172.21.15.92:41736) 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f5845640 1 -- 172.21.15.92:0/1617529969 learned_addr learned my addr 172.21.15.92:0/1617529969 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f5845640 1 -- 172.21.15.92:0/1617529969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f019b910 con 0x7f48f0106320 2024-09-11T22:04:42.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f5845640 1 --2- 172.21.15.92:0/1617529969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f019b3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009940 tx=0x7f48ec032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48e6ffd640 1 -- 172.21.15.92:0/1617529969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48ec042070 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48e6ffd640 1 -- 172.21.15.92:0/1617529969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f48ec030c80 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f019bba0 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48e6ffd640 1 -- 172.21.15.92:0/1617529969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48ec037040 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.001+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f019e890 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.002+0000 7f48e6ffd640 1 -- 172.21.15.92:0/1617529969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f48ec045070 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.002+0000 7f48e6ffd640 1 -- 172.21.15.92:0/1617529969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f48ec0479e0 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.002+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f010aa50 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.004+0000 7f48e6ffd640 1 -- 172.21.15.92:0/1617529969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f48ec03c5f0 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.065+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f48f01067a0 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.066+0000 7f48e6ffd640 1 -- 172.21.15.92:0/1617529969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f48f01067a0 con 0x7f48f0106320 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.067+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 msgr2=0x7f48f019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:42.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.067+0000 7f48f7aa9640 1 --2- 172.21.15.92:0/1617529969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f019b3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009940 tx=0x7f48ec032e10 comp rx=0 tx=0).stop 2024-09-11T22:04:42.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.067+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 shutdown_connections 2024-09-11T22:04:42.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.067+0000 7f48f7aa9640 1 --2- 172.21.15.92:0/1617529969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48f0106320 0x7f48f019b3d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:42.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.067+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 >> 172.21.15.92:0/1617529969 conn(0x7f48f0101a90 msgr2=0x7f48f0102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:42.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.067+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 shutdown_connections 2024-09-11T22:04:42.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:42.067+0000 7f48f7aa9640 1 -- 172.21.15.92:0/1617529969 wait complete. 2024-09-11T22:04:42.656 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (2/15)... 2024-09-11T22:04:43.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:42 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:42.917+0000 7f6e7c89f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:04:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: Activating manager daemon x 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: mgrmap e2: x(active, starting, since 0.00299793s) 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: Manager daemon x is now available 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' 2024-09-11T22:04:43.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:43 smithi092 ceph-mon[31173]: from='mgr.14100 172.21.15.92:0/3655694744' entity='mgr.x' 2024-09-11T22:04:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:44 smithi092 ceph-mon[31173]: mgrmap e3: x(active, since 1.00718s) 2024-09-11T22:04:45.813 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:45.813 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:04:45.813 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:04:45.813 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:04:45.813 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:04:45.813 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:04:45.813 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:04:45.815 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:04:45.815 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:45.815 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:04:45.815 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:45.815 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:04:45.815 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:04:45.816 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:04:45.816 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:04:45.816 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:04:45.816 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:04:45.825 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:45.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:04:29:869139+0000", 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:04:45.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:04:29.869766+0000", 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:04:45.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.978+0000 7fc2fefb9640 1 Processor -- start 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.978+0000 7fc2fefb9640 1 -- start start 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.978+0000 7fc2fefb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f8107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.978+0000 7fc2fefb9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8107df0 con 0x7fc2f8107420 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.979+0000 7fc2fcd55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f8107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.979+0000 7fc2fcd55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f8107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41798/0 (socket says 172.21.15.92:41798) 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.979+0000 7fc2fcd55640 1 -- 172.21.15.92:0/667401122 learned_addr learned my addr 172.21.15.92:0/667401122 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.979+0000 7fc2fcd55640 1 -- 172.21.15.92:0/667401122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f8108620 con 0x7fc2f8107420 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fcd55640 1 --2- 172.21.15.92:0/667401122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f8107820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009820 tx=0x7fc2e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f36ac41001921d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2f77fe640 1 -- 172.21.15.92:0/667401122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2e8037070 con 0x7fc2f8107420 2024-09-11T22:04:45.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2f77fe640 1 -- 172.21.15.92:0/667401122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc2e80328d0 con 0x7fc2f8107420 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2f77fe640 1 -- 172.21.15.92:0/667401122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2e80307d0 con 0x7fc2f8107420 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fefb9640 1 -- 172.21.15.92:0/667401122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 msgr2=0x7fc2f8107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fefb9640 1 --2- 172.21.15.92:0/667401122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f8107820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009820 tx=0x7fc2e802a270 comp rx=0 tx=0).stop 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fefb9640 1 -- 172.21.15.92:0/667401122 shutdown_connections 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fefb9640 1 --2- 172.21.15.92:0/667401122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f8107820 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fefb9640 1 -- 172.21.15.92:0/667401122 >> 172.21.15.92:0/667401122 conn(0x7fc2f8102bb0 msgr2=0x7fc2f8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fefb9640 1 -- 172.21.15.92:0/667401122 shutdown_connections 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.980+0000 7fc2fefb9640 1 -- 172.21.15.92:0/667401122 wait complete. 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.981+0000 7fc2fefb9640 1 Processor -- start 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.981+0000 7fc2fefb9640 1 -- start start 2024-09-11T22:04:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.981+0000 7fc2fefb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.981+0000 7fc2fefb9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f810b9d0 con 0x7fc2f8107420 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.981+0000 7fc2fcd55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.981+0000 7fc2fcd55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41808/0 (socket says 172.21.15.92:41808) 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.981+0000 7fc2fcd55640 1 -- 172.21.15.92:0/2194059955 learned_addr learned my addr 172.21.15.92:0/2194059955 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.982+0000 7fc2fcd55640 1 -- 172.21.15.92:0/2194059955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f819b8d0 con 0x7fc2f8107420 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.982+0000 7fc2fcd55640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f819b390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009940 tx=0x7fc2e8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.982+0000 7fc2f5ffb640 1 -- 172.21.15.92:0/2194059955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2e8042070 con 0x7fc2f8107420 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.982+0000 7fc2f5ffb640 1 -- 172.21.15.92:0/2194059955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc2e8030c80 con 0x7fc2f8107420 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.982+0000 7fc2f5ffb640 1 -- 172.21.15.92:0/2194059955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2e8037040 con 0x7fc2f8107420 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.982+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f819bb60 con 0x7fc2f8107420 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.982+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f819e850 con 0x7fc2f8107420 2024-09-11T22:04:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.983+0000 7fc2f5ffb640 1 -- 172.21.15.92:0/2194059955 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc2e8045070 con 0x7fc2f8107420 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.983+0000 7fc2f5ffb640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fc2e4038f70 0x7fc2e403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.983+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f810bb50 con 0x7fc2f8107420 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.983+0000 7fc2f5ffb640 1 -- 172.21.15.92:0/2194059955 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2e806be00 con 0x7fc2f8107420 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.983+0000 7fc2f7fff640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fc2e4038f70 0x7fc2e403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.984+0000 7fc2f7fff640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fc2e4038f70 0x7fc2e403b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00098c0 tx=0x7fc2e0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:44.988+0000 7fc2f5ffb640 1 -- 172.21.15.92:0/2194059955 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2e806b820 con 0x7fc2f8107420 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.217+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc2f819eec0 con 0x7fc2f8107420 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.218+0000 7fc2f5ffb640 1 -- 172.21.15.92:0/2194059955 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fc2e802ed70 con 0x7fc2f8107420 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fc2e4038f70 msgr2=0x7fc2e403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fc2e4038f70 0x7fc2e403b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00098c0 tx=0x7fc2e0006db0 comp rx=0 tx=0).stop 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 msgr2=0x7fc2f819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f819b390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009940 tx=0x7fc2e8032e10 comp rx=0 tx=0).stop 2024-09-11T22:04:45.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 shutdown_connections 2024-09-11T22:04:45.833 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fc2e4038f70 0x7fc2e403b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:45.833 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 --2- 172.21.15.92:0/2194059955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8107420 0x7fc2f819b390 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:45.833 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 >> 172.21.15.92:0/2194059955 conn(0x7fc2f8102bb0 msgr2=0x7fc2f81036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:45.833 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 shutdown_connections 2024-09-11T22:04:45.833 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:45.220+0000 7fc2fefb9640 1 -- 172.21.15.92:0/2194059955 wait complete. 2024-09-11T22:04:45.833 INFO:teuthology.orchestra.run.smithi092.stdout:mgr is available 2024-09-11T22:04:46.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:45 smithi092 ceph-mon[31173]: mgrmap e4: x(active, since 2s) 2024-09-11T22:04:46.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:45 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2194059955' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:04:46.981 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T22:04:46.982 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.190+0000 7fac180a1640 1 Processor -- start 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.191+0000 7fac180a1640 1 -- start start 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.191+0000 7fac180a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.191+0000 7fac180a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac1007c870 con 0x7fac1007de40 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.191+0000 7fac15e3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.191+0000 7fac15e3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59454/0 (socket says 172.21.15.92:59454) 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.191+0000 7fac15e3d640 1 -- 172.21.15.92:0/3778729523 learned_addr learned my addr 172.21.15.92:0/3778729523 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.191+0000 7fac15e3d640 1 -- 172.21.15.92:0/3778729523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1007c9f0 con 0x7fac1007de40 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.192+0000 7fac15e3d640 1 --2- 172.21.15.92:0/3778729523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1007c2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac00009820 tx=0x7fac0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2414b92f0b02be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:46.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.192+0000 7fac14e3b640 1 -- 172.21.15.92:0/3778729523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac00037070 con 0x7fac1007de40 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.192+0000 7fac14e3b640 1 -- 172.21.15.92:0/3778729523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fac000328d0 con 0x7fac1007de40 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.192+0000 7fac14e3b640 1 -- 172.21.15.92:0/3778729523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac000307d0 con 0x7fac1007de40 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 -- 172.21.15.92:0/3778729523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 msgr2=0x7fac1007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 --2- 172.21.15.92:0/3778729523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1007c2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac00009820 tx=0x7fac0002a270 comp rx=0 tx=0).stop 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 -- 172.21.15.92:0/3778729523 shutdown_connections 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 --2- 172.21.15.92:0/3778729523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1007c2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 -- 172.21.15.92:0/3778729523 >> 172.21.15.92:0/3778729523 conn(0x7fac10102bb0 msgr2=0x7fac10104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 -- 172.21.15.92:0/3778729523 shutdown_connections 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 -- 172.21.15.92:0/3778729523 wait complete. 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 Processor -- start 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.193+0000 7fac180a1640 1 -- start start 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac180a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac180a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac1010da30 con 0x7fac1007de40 2024-09-11T22:04:46.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac15e3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac15e3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1019d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59460/0 (socket says 172.21.15.92:59460) 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac15e3d640 1 -- 172.21.15.92:0/934519097 learned_addr learned my addr 172.21.15.92:0/934519097 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac15e3d640 1 -- 172.21.15.92:0/934519097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1019daa0 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac15e3d640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1019d560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac00032e10 tx=0x7fac00032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac06ffd640 1 -- 172.21.15.92:0/934519097 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac00042070 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac06ffd640 1 -- 172.21.15.92:0/934519097 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fac00030c80 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac06ffd640 1 -- 172.21.15.92:0/934519097 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac00037040 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1019dd30 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.194+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac101a0a20 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.195+0000 7fac06ffd640 1 -- 172.21.15.92:0/934519097 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fac00045070 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.195+0000 7fac06ffd640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fabfc038f20 0x7fabfc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.196+0000 7fac06ffd640 1 -- 172.21.15.92:0/934519097 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac0006beb0 con 0x7fac1007de40 2024-09-11T22:04:46.985 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.196+0000 7fac1563c640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fabfc038f20 0x7fabfc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.196+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1007c320 con 0x7fac1007de40 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.196+0000 7fac1563c640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fabfc038f20 0x7fabfc03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0098c0 tx=0x7fac0c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.200+0000 7fac06ffd640 1 -- 172.21.15.92:0/934519097 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac00045360 con 0x7fac1007de40 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.367+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fac101a2030 con 0x7fac1007de40 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.368+0000 7fac06ffd640 1 -- 172.21.15.92:0/934519097 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fac0006b8d0 con 0x7fac1007de40 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.369+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fabfc038f20 msgr2=0x7fabfc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.369+0000 7fac180a1640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fabfc038f20 0x7fabfc03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0098c0 tx=0x7fac0c006db0 comp rx=0 tx=0).stop 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 msgr2=0x7fac1019d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1019d560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac00032e10 tx=0x7fac00032e40 comp rx=0 tx=0).stop 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 shutdown_connections 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7fabfc038f20 0x7fabfc03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:46.986 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 --2- 172.21.15.92:0/934519097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac1007de40 0x7fac1019d560 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:46.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 >> 172.21.15.92:0/934519097 conn(0x7fac10102bb0 msgr2=0x7fac10103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:46.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 shutdown_connections 2024-09-11T22:04:46.987 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:46.370+0000 7fac180a1640 1 -- 172.21.15.92:0/934519097 wait complete. 2024-09-11T22:04:46.987 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling cephadm module... 2024-09-11T22:04:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:46 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/934519097' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T22:04:48.245 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:47 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2826395877' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T22:04:48.246 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:47 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: ignoring --setuser ceph since I am not root 2024-09-11T22:04:48.246 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:47 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: ignoring --setgroup ceph since I am not root 2024-09-11T22:04:48.246 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:48.072+0000 7ff676dd3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:04:48.246 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:48.129+0000 7ff676dd3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:04:48.519 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:48.278+0000 7ff676dd3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:04:48.519 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:48.343+0000 7ff676dd3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:04:48.519 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:48.492+0000 7ff676dd3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2ad6153640 1 Processor -- start 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2ad6153640 1 -- start start 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2ad6153640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad010a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2ad6153640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad007bdd0 con 0x7f2ad0107f00 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad010a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad010a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59476/0 (socket says 172.21.15.92:59476) 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2acf7fe640 1 -- 172.21.15.92:0/2179231492 learned_addr learned my addr 172.21.15.92:0/2179231492 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.395+0000 7f2acf7fe640 1 -- 172.21.15.92:0/2179231492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad010a830 con 0x7f2ad0107f00 2024-09-11T22:04:48.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.396+0000 7f2acf7fe640 1 --2- 172.21.15.92:0/2179231492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad010a2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009820 tx=0x7f2ac002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf528e278c9e6101 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.396+0000 7f2ace7fc640 1 -- 172.21.15.92:0/2179231492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2ad0107f00 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.396+0000 7f2ace7fc640 1 -- 172.21.15.92:0/2179231492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2ac00328d0 con 0x7f2ad0107f00 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.396+0000 7f2ace7fc640 1 -- 172.21.15.92:0/2179231492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac00307d0 con 0x7f2ad0107f00 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 -- 172.21.15.92:0/2179231492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 msgr2=0x7f2ad010a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 --2- 172.21.15.92:0/2179231492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad010a2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009820 tx=0x7f2ac002a270 comp rx=0 tx=0).stop 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 -- 172.21.15.92:0/2179231492 shutdown_connections 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 --2- 172.21.15.92:0/2179231492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad010a2f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 -- 172.21.15.92:0/2179231492 >> 172.21.15.92:0/2179231492 conn(0x7f2ad0101ae0 msgr2=0x7f2ad0103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 -- 172.21.15.92:0/2179231492 shutdown_connections 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 -- 172.21.15.92:0/2179231492 wait complete. 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.397+0000 7f2ad6153640 1 Processor -- start 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2ad6153640 1 -- start start 2024-09-11T22:04:48.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2ad6153640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2ad6153640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010b7b0 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59480/0 (socket says 172.21.15.92:59480) 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2acf7fe640 1 -- 172.21.15.92:0/2826395877 learned_addr learned my addr 172.21.15.92:0/2826395877 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2acf7fe640 1 -- 172.21.15.92:0/2826395877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad019b890 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2acf7fe640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad019b350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0032e10 tx=0x7f2ac0032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0042070 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad019bb20 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.398+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2ac0030c80 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.399+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0037040 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.399+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad019e810 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.399+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f2ac0045070 con 0x7f2ad0107f00 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.400+0000 7f2accff9640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f2ab8038f20 0x7f2ab803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:48.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.400+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad0108360 con 0x7f2ad0107f00 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.400+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2ac006bfe0 con 0x7f2ad0107f00 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.400+0000 7f2aceffd640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f2ab8038f20 0x7f2ab803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.400+0000 7f2aceffd640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f2ab8038f20 0x7f2ab803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40098c0 tx=0x7f2ac4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.404+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ac0045360 con 0x7f2ad0107f00 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.615+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2ad019ee60 con 0x7f2ad0107f00 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.929+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2ac003bde0 con 0x7f2ad0107f00 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.930+0000 7f2accff9640 1 -- 172.21.15.92:0/2826395877 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f2ac003cc20 con 0x7f2ad0107f00 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f2ab8038f20 msgr2=0x7f2ab803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f2ab8038f20 0x7f2ab803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40098c0 tx=0x7f2ac4006db0 comp rx=0 tx=0).stop 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 msgr2=0x7f2ad019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad019b350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0032e10 tx=0x7f2ac0032e40 comp rx=0 tx=0).stop 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 shutdown_connections 2024-09-11T22:04:48.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f2ab8038f20 0x7f2ab803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:48.558 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 --2- 172.21.15.92:0/2826395877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ad0107f00 0x7f2ad019b350 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:48.558 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 >> 172.21.15.92:0/2826395877 conn(0x7f2ad0101ae0 msgr2=0x7f2ad0102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:48.558 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 shutdown_connections 2024-09-11T22:04:48.558 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:47.932+0000 7f2ad6153640 1 -- 172.21.15.92:0/2826395877 wait complete. 2024-09-11T22:04:48.806 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:48.638+0000 7ff676dd3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:04:49.175 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2826395877' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T22:04:49.175 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-mon[31173]: mgrmap e5: x(active, since 5s) 2024-09-11T22:04:49.175 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:48.958+0000 7ff676dd3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:04:49.175 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:49.022+0000 7ff676dd3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:04:49.477 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:49.216+0000 7ff676dd3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:04:49.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.968+0000 7f7ec9db0640 1 Processor -- start 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.968+0000 7f7ec9db0640 1 -- start start 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.968+0000 7f7ec9db0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec410a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.968+0000 7f7ec9db0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4075820 con 0x7f7ec4107ef0 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.968+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec410a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.968+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec410a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59498/0 (socket says 172.21.15.92:59498) 2024-09-11T22:04:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.968+0000 7f7ec37fe640 1 -- 172.21.15.92:0/1489845337 learned_addr learned my addr 172.21.15.92:0/1489845337 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.969+0000 7f7ec37fe640 1 -- 172.21.15.92:0/1489845337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec410a820 con 0x7f7ec4107ef0 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.969+0000 7f7ec37fe640 1 --2- 172.21.15.92:0/1489845337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec410a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009820 tx=0x7f7eb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7526a1db21ff9ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.969+0000 7f7ec27fc640 1 -- 172.21.15.92:0/1489845337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eb4037070 con 0x7f7ec4107ef0 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.969+0000 7f7ec27fc640 1 -- 172.21.15.92:0/1489845337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7eb40328d0 con 0x7f7ec4107ef0 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.969+0000 7f7ec27fc640 1 -- 172.21.15.92:0/1489845337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eb40307d0 con 0x7f7ec4107ef0 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.970+0000 7f7ec9db0640 1 -- 172.21.15.92:0/1489845337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 msgr2=0x7f7ec410a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.970+0000 7f7ec9db0640 1 --2- 172.21.15.92:0/1489845337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec410a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009820 tx=0x7f7eb402a270 comp rx=0 tx=0).stop 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.970+0000 7f7ec9db0640 1 -- 172.21.15.92:0/1489845337 shutdown_connections 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.970+0000 7f7ec9db0640 1 --2- 172.21.15.92:0/1489845337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec410a2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.970+0000 7f7ec9db0640 1 -- 172.21.15.92:0/1489845337 >> 172.21.15.92:0/1489845337 conn(0x7f7ec4101a90 msgr2=0x7f7ec4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.970+0000 7f7ec9db0640 1 -- 172.21.15.92:0/1489845337 shutdown_connections 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.970+0000 7f7ec9db0640 1 -- 172.21.15.92:0/1489845337 wait complete. 2024-09-11T22:04:49.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.971+0000 7f7ec9db0640 1 Processor -- start 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.971+0000 7f7ec9db0640 1 -- start start 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.971+0000 7f7ec9db0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.971+0000 7f7ec9db0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410b800 con 0x7f7ec4107ef0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.971+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.971+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59512/0 (socket says 172.21.15.92:59512) 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.971+0000 7f7ec37fe640 1 -- 172.21.15.92:0/2853146004 learned_addr learned my addr 172.21.15.92:0/2853146004 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.972+0000 7f7ec37fe640 1 -- 172.21.15.92:0/2853146004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec419dac0 con 0x7f7ec4107ef0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.972+0000 7f7ec37fe640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec419d580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40097f0 tx=0x7f7eb4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.972+0000 7f7ec0ff9640 1 -- 172.21.15.92:0/2853146004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eb4037040 con 0x7f7ec4107ef0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.972+0000 7f7ec0ff9640 1 -- 172.21.15.92:0/2853146004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7eb4032bb0 con 0x7f7ec4107ef0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.972+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec419dd50 con 0x7f7ec4107ef0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.972+0000 7f7ec0ff9640 1 -- 172.21.15.92:0/2853146004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eb403a390 con 0x7f7ec4107ef0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.972+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec41a0a40 con 0x7f7ec4107ef0 2024-09-11T22:04:49.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.973+0000 7f7ec0ff9640 1 -- 172.21.15.92:0/2853146004 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f7eb403a580 con 0x7f7ec4107ef0 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.973+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec4108350 con 0x7f7ec4107ef0 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.973+0000 7f7ec0ff9640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 0x7f7eac03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.973+0000 7f7ec0ff9640 1 -- 172.21.15.92:0/2853146004 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7eb406c110 con 0x7f7ec4107ef0 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.973+0000 7f7ec2ffd640 1 -- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 msgr2=0x7f7eac03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1519869556 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.973+0000 7f7ec2ffd640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 0x7f7eac03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:48.978+0000 7f7ec0ff9640 1 -- 172.21.15.92:0/2853146004 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7eb4040210 con 0x7f7ec4107ef0 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.174+0000 7f7ec2ffd640 1 -- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 msgr2=0x7f7eac03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1519869556 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.174+0000 7f7ec2ffd640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 0x7f7eac03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.197+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7ec41a10c0 con 0x7f7ec4107ef0 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.197+0000 7f7ec0ff9640 1 -- 172.21.15.92:0/2853146004 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f7eb4031560 con 0x7f7ec4107ef0 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 msgr2=0x7f7eac03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 0x7f7eac03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:49.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 msgr2=0x7f7ec419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec419d580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40097f0 tx=0x7f7eb4030d00 comp rx=0 tx=0).stop 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 shutdown_connections 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7eac038fc0 0x7f7eac03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 --2- 172.21.15.92:0/2853146004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec4107ef0 0x7f7ec419d580 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 >> 172.21.15.92:0/2853146004 conn(0x7f7ec4101a90 msgr2=0x7f7ec4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 shutdown_connections 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:49.200+0000 7f7ec9db0640 1 -- 172.21.15.92:0/2853146004 wait complete. 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-09-11T22:04:49.749 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 5... 2024-09-11T22:04:50.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:49 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2853146004' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T22:04:50.332 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:50.123+0000 7ff676dd3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:04:50.332 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:50.193+0000 7ff676dd3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:04:51.011 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:50.894+0000 7ff676dd3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:04:51.011 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.010+0000 7ff676dd3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:04:51.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.152+0000 7ff676dd3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:04:51.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.212+0000 7ff676dd3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:04:51.647 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.362+0000 7ff676dd3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:04:51.647 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.429+0000 7ff676dd3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:04:51.647 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.489+0000 7ff676dd3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:04:51.647 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.646+0000 7ff676dd3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:04:51.917 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.710+0000 7ff676dd3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:04:51.917 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.775+0000 7ff676dd3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:04:51.918 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.857+0000 7ff676dd3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:04:51.918 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.916+0000 7ff676dd3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:04:52.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:51.976+0000 7ff676dd3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:04:53.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:52.788+0000 7ff676dd3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: Active manager daemon x restarted 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: Activating manager daemon x 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: mgrmap e6: x(active, starting, since 0.00417077s) 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: Manager daemon x is now available 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:04:53.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:04:53.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:04:53.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:04:53.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:52 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:04:54.084 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:53 smithi092 ceph-mon[31173]: Found migration_current of "None". Setting to last migration. 2024-09-11T22:04:54.085 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:53 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:04:54.085 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:53 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:04:54.085 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:53 smithi092 ceph-mon[31173]: mgrmap e7: x(active, since 1.00576s) 2024-09-11T22:04:54.335 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:54.308+0000 7ff5e3938640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:04:54] ENGINE Error in HTTPServer.serve 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: Traceback (most recent call last): 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self._connections.run(self.expiration_interval) 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self._run(expiration_interval) 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: s = self.context.wrap_socket( 2024-09-11T22:04:54.336 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:04:54.337 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: return self.sslsocket_class._create( 2024-09-11T22:04:54.337 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:04:54.337 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self.do_handshake() 2024-09-11T22:04:54.337 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:04:54.337 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self._sslobj.do_handshake() 2024-09-11T22:04:54.337 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:04:54.337 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:04:54.337 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:04:54.344 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T22:04:54.345 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T22:04:54.345 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:04:54.345 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.125+0000 7f7eb6223640 1 Processor -- start 2024-09-11T22:04:54.345 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.125+0000 7f7eb6223640 1 -- start start 2024-09-11T22:04:54.345 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.125+0000 7f7eb6223640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb010a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.125+0000 7f7eb6223640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00757a0 con 0x7f7eb0107f60 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.125+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb010a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.125+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb010a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59520/0 (socket says 172.21.15.92:59520) 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.125+0000 7f7eaf7fe640 1 -- 172.21.15.92:0/3961086729 learned_addr learned my addr 172.21.15.92:0/3961086729 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.126+0000 7f7eaf7fe640 1 -- 172.21.15.92:0/3961086729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb010a890 con 0x7f7eb0107f60 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.126+0000 7f7eaf7fe640 1 --2- 172.21.15.92:0/3961086729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb010a350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009820 tx=0x7f7ea002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2bb8486df216af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eae7fc640 1 -- 172.21.15.92:0/3961086729 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea0037070 con 0x7f7eb0107f60 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eae7fc640 1 -- 172.21.15.92:0/3961086729 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7ea00328d0 con 0x7f7eb0107f60 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eae7fc640 1 -- 172.21.15.92:0/3961086729 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea00307d0 con 0x7f7eb0107f60 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eb6223640 1 -- 172.21.15.92:0/3961086729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 msgr2=0x7f7eb010a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eb6223640 1 --2- 172.21.15.92:0/3961086729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb010a350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009820 tx=0x7f7ea002a270 comp rx=0 tx=0).stop 2024-09-11T22:04:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eb6223640 1 -- 172.21.15.92:0/3961086729 shutdown_connections 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eb6223640 1 --2- 172.21.15.92:0/3961086729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb010a350 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eb6223640 1 -- 172.21.15.92:0/3961086729 >> 172.21.15.92:0/3961086729 conn(0x7f7eb0101ac0 msgr2=0x7f7eb0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eb6223640 1 -- 172.21.15.92:0/3961086729 shutdown_connections 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.127+0000 7f7eb6223640 1 -- 172.21.15.92:0/3961086729 wait complete. 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.128+0000 7f7eb6223640 1 Processor -- start 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.128+0000 7f7eb6223640 1 -- start start 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.128+0000 7f7eb6223640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.128+0000 7f7eb6223640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010b7b0 con 0x7f7eb0107f60 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59536/0 (socket says 172.21.15.92:59536) 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eaf7fe640 1 -- 172.21.15.92:0/1282701920 learned_addr learned my addr 172.21.15.92:0/1282701920 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eaf7fe640 1 -- 172.21.15.92:0/1282701920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb019b8a0 con 0x7f7eb0107f60 2024-09-11T22:04:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eaf7fe640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb019b360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0032e10 tx=0x7f7ea0032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea0042070 con 0x7f7eb0107f60 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7ea0030c80 con 0x7f7eb0107f60 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb019bb30 con 0x7f7eb0107f60 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea0037040 con 0x7f7eb0107f60 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.129+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb019e820 con 0x7f7eb0107f60 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.130+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f7ea0045070 con 0x7f7eb0107f60 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.130+0000 7f7eacff9640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 0x7f7e9c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.130+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 --> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7eb01083c0 con 0x7f7e9c038fc0 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.130+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ea006bee0 con 0x7f7eb0107f60 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.130+0000 7f7eaeffd640 1 -- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 msgr2=0x7f7e9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1519869556 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.130+0000 7f7eaeffd640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 0x7f7e9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:04:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.331+0000 7f7eaeffd640 1 -- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 msgr2=0x7f7e9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1519869556 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.331+0000 7f7eaeffd640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 0x7f7e9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.732+0000 7f7eaeffd640 1 -- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 msgr2=0x7f7e9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1519869556 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:50.732+0000 7f7eaeffd640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 0x7f7e9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:51.533+0000 7f7eaeffd640 1 -- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 msgr2=0x7f7e9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1519869556 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:51.533+0000 7f7eaeffd640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 0x7f7e9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:52.793+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f7ea006a9a0 con 0x7f7eb0107f60 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:52.793+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 msgr2=0x7f7e9c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:52.793+0000 7f7eacff9640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 0x7f7e9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.793+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f7ea003c610 con 0x7f7eb0107f60 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.793+0000 7f7eacff9640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7e9c03c700 0x7f7e9c03eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.793+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7ea002eb70 con 0x7f7e9c03c700 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.794+0000 7f7eaeffd640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7e9c03c700 0x7f7e9c03eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.794+0000 7f7eaeffd640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7e9c03c700 0x7f7e9c03eaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7e98003980 tx=0x7f7e980092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.795+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7ea002eb70 con 0x7f7e9c03c700 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.807+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7eb0102cd0 con 0x7f7e9c03c700 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.807+0000 7f7eacff9640 1 -- 172.21.15.92:0/1282701920 <== mgr.14118 v2:172.21.15.92:6800/733069068 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7eb0102cd0 con 0x7f7e9c03c700 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7e9c03c700 msgr2=0x7f7e9c03eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7e9c03c700 0x7f7e9c03eaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7e98003980 tx=0x7f7e980092b0 comp rx=0 tx=0).stop 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 msgr2=0x7f7eb019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb019b360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0032e10 tx=0x7f7ea0032e40 comp rx=0 tx=0).stop 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 shutdown_connections 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7e9c03c700 0x7f7e9c03eaf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:6800/1519869556,v1:172.21.15.92:6801/1519869556] conn(0x7f7e9c038fc0 0x7f7e9c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 --2- 172.21.15.92:0/1282701920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7eb0107f60 0x7f7eb019b360 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 >> 172.21.15.92:0/1282701920 conn(0x7f7eb0101ac0 msgr2=0x7f7eb01025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:54.351 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 shutdown_connections 2024-09-11T22:04:54.351 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:53.808+0000 7f7eb6223640 1 -- 172.21.15.92:0/1282701920 wait complete. 2024-09-11T22:04:54.351 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 5 is available 2024-09-11T22:04:54.351 INFO:teuthology.orchestra.run.smithi092.stdout:Setting orchestrator backend to cephadm... 2024-09-11T22:04:54.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-mon[31173]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T22:04:54.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-mon[31173]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T22:04:54.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:54 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1e577f640 1 Processor -- start 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1e577f640 1 -- start start 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1e577f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e0107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1e577f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0107df0 con 0x7fe1e0107420 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e0107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e0107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59586/0 (socket says 172.21.15.92:59586) 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1deffd640 1 -- 172.21.15.92:0/3251972827 learned_addr learned my addr 172.21.15.92:0/3251972827 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.708+0000 7fe1deffd640 1 -- 172.21.15.92:0/3251972827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0108620 con 0x7fe1e0107420 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.709+0000 7fe1deffd640 1 --2- 172.21.15.92:0/3251972827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e0107820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009820 tx=0x7fe1d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4a853f99bbaacb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.709+0000 7fe1de7fc640 1 -- 172.21.15.92:0/3251972827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d0037070 con 0x7fe1e0107420 2024-09-11T22:04:55.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.709+0000 7fe1de7fc640 1 -- 172.21.15.92:0/3251972827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe1d00328d0 con 0x7fe1e0107420 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.709+0000 7fe1de7fc640 1 -- 172.21.15.92:0/3251972827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d00307d0 con 0x7fe1e0107420 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 -- 172.21.15.92:0/3251972827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 msgr2=0x7fe1e0107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 --2- 172.21.15.92:0/3251972827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e0107820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009820 tx=0x7fe1d002a270 comp rx=0 tx=0).stop 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 -- 172.21.15.92:0/3251972827 shutdown_connections 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 --2- 172.21.15.92:0/3251972827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e0107820 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 -- 172.21.15.92:0/3251972827 >> 172.21.15.92:0/3251972827 conn(0x7fe1e0102bb0 msgr2=0x7fe1e0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 -- 172.21.15.92:0/3251972827 shutdown_connections 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 -- 172.21.15.92:0/3251972827 wait complete. 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.710+0000 7fe1e577f640 1 Processor -- start 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1e577f640 1 -- start start 2024-09-11T22:04:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1e577f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1e577f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e010b9d0 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59602/0 (socket says 172.21.15.92:59602) 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1deffd640 1 -- 172.21.15.92:0/3109582283 learned_addr learned my addr 172.21.15.92:0/3109582283 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1deffd640 1 -- 172.21.15.92:0/3109582283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e019b8d0 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1deffd640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e019b390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009940 tx=0x7fe1d0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d0042070 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe1d0030c80 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e019bb60 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.711+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d0037040 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.712+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e019e850 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.712+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fe1d0045070 con 0x7fe1e0107420 2024-09-11T22:04:55.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.712+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e010bb50 con 0x7fe1e0107420 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.713+0000 7fe1dcff9640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fe1cc038e60 0x7fe1cc03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.713+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1d006bd00 con 0x7fe1e0107420 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.713+0000 7fe1d67fc640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fe1cc038e60 0x7fe1cc03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.713+0000 7fe1d67fc640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fe1cc038e60 0x7fe1cc03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80098c0 tx=0x7fe1c8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.718+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1d0045360 con 0x7fe1e0107420 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.853+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe1d0072080 con 0x7fe1e0107420 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.902+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fe1e0103dd0 con 0x7fe1cc038e60 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.909+0000 7fe1dcff9640 1 -- 172.21.15.92:0/3109582283 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe1e0103dd0 con 0x7fe1cc038e60 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.911+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fe1cc038e60 msgr2=0x7fe1cc03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.911+0000 7fe1e577f640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fe1cc038e60 0x7fe1cc03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80098c0 tx=0x7fe1c8006db0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.911+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 msgr2=0x7fe1e019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.911+0000 7fe1e577f640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e019b390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009940 tx=0x7fe1d0032e10 comp rx=0 tx=0).stop 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.912+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 shutdown_connections 2024-09-11T22:04:55.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.912+0000 7fe1e577f640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fe1cc038e60 0x7fe1cc03b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.912+0000 7fe1e577f640 1 --2- 172.21.15.92:0/3109582283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e0107420 0x7fe1e019b390 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.912+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 >> 172.21.15.92:0/3109582283 conn(0x7fe1e0102bb0 msgr2=0x7fe1e01036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.912+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 shutdown_connections 2024-09-11T22:04:55.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:54.912+0000 7fe1e577f640 1 -- 172.21.15.92:0/3109582283 wait complete. 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: [11/Sep/2024:22:04:54] ENGINE Bus STARTING 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: [11/Sep/2024:22:04:54] ENGINE Serving on https://172.21.15.92:7150 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: [11/Sep/2024:22:04:54] ENGINE Error in HTTPServer.serve 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout: Traceback (most recent call last): 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._run(expiration_interval) 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:04:56.032 INFO:journalctl@ceph.mon.a.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: s = self.context.wrap_socket( 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: return self.sslsocket_class._create( 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: self.do_handshake() 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._sslobj.do_handshake() 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: [11/Sep/2024:22:04:54] ENGINE Serving on http://172.21.15.92:8765 2024-09-11T22:04:56.033 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: [11/Sep/2024:22:04:54] ENGINE Bus STARTED 2024-09-11T22:04:56.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: mgrmap e8: x(active, since 2s) 2024-09-11T22:04:56.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:04:56.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:04:56.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:55 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:04:56.599 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T22:04:56.599 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.854+0000 7f9a76abd640 1 Processor -- start 2024-09-11T22:04:56.599 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.854+0000 7f9a76abd640 1 -- start start 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.854+0000 7f9a76abd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a70107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.854+0000 7f9a76abd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70107dd0 con 0x7f9a70107400 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.854+0000 7f9a74859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a70107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.854+0000 7f9a74859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a70107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39006/0 (socket says 172.21.15.92:39006) 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.854+0000 7f9a74859640 1 -- 172.21.15.92:0/861600041 learned_addr learned my addr 172.21.15.92:0/861600041 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.855+0000 7f9a74859640 1 -- 172.21.15.92:0/861600041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a70108600 con 0x7f9a70107400 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.855+0000 7f9a74859640 1 --2- 172.21.15.92:0/861600041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a70107800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009820 tx=0x7f9a6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e264cb31053d008 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.855+0000 7f9a6f7fe640 1 -- 172.21.15.92:0/861600041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a64037070 con 0x7f9a70107400 2024-09-11T22:04:56.600 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.855+0000 7f9a6f7fe640 1 -- 172.21.15.92:0/861600041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9a640328d0 con 0x7f9a70107400 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.855+0000 7f9a6f7fe640 1 -- 172.21.15.92:0/861600041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a640307d0 con 0x7f9a70107400 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.856+0000 7f9a76abd640 1 -- 172.21.15.92:0/861600041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 msgr2=0x7f9a70107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.856+0000 7f9a76abd640 1 --2- 172.21.15.92:0/861600041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a70107800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009820 tx=0x7f9a6402a270 comp rx=0 tx=0).stop 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.856+0000 7f9a76abd640 1 -- 172.21.15.92:0/861600041 shutdown_connections 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.856+0000 7f9a76abd640 1 --2- 172.21.15.92:0/861600041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a70107800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.856+0000 7f9a76abd640 1 -- 172.21.15.92:0/861600041 >> 172.21.15.92:0/861600041 conn(0x7f9a70102bd0 msgr2=0x7f9a70104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.856+0000 7f9a76abd640 1 -- 172.21.15.92:0/861600041 shutdown_connections 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.856+0000 7f9a76abd640 1 -- 172.21.15.92:0/861600041 wait complete. 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a76abd640 1 Processor -- start 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a76abd640 1 -- start start 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a76abd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a7007ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a76abd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010b9b0 con 0x7f9a70107400 2024-09-11T22:04:56.601 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a74859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a7007ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a74859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a7007ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39018/0 (socket says 172.21.15.92:39018) 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a74859640 1 -- 172.21.15.92:0/3336247431 learned_addr learned my addr 172.21.15.92:0/3336247431 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a74859640 1 -- 172.21.15.92:0/3336247431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7007cf40 con 0x7f9a70107400 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.857+0000 7f9a74859640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a7007ca00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a640097f0 tx=0x7f9a64030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.858+0000 7f9a6dffb640 1 -- 172.21.15.92:0/3336247431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a64037040 con 0x7f9a70107400 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.858+0000 7f9a6dffb640 1 -- 172.21.15.92:0/3336247431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9a64032bb0 con 0x7f9a70107400 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.858+0000 7f9a6dffb640 1 -- 172.21.15.92:0/3336247431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a6403a390 con 0x7f9a70107400 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.858+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7007b130 con 0x7f9a70107400 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.858+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7007b670 con 0x7f9a70107400 2024-09-11T22:04:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.859+0000 7f9a6dffb640 1 -- 172.21.15.92:0/3336247431 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9a6403b3f0 con 0x7f9a70107400 2024-09-11T22:04:56.603 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.859+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70107880 con 0x7f9a70107400 2024-09-11T22:04:56.603 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.859+0000 7f9a6dffb640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9a60038fc0 0x7f9a6003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:56.603 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.859+0000 7f9a6dffb640 1 -- 172.21.15.92:0/3336247431 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9a6406c190 con 0x7f9a70107400 2024-09-11T22:04:56.603 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.859+0000 7f9a6ffff640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9a60038fc0 0x7f9a6003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:56.603 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.860+0000 7f9a6ffff640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9a60038fc0 0x7f9a6003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0098c0 tx=0x7f9a5c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:56.603 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:55.864+0000 7f9a6dffb640 1 -- 172.21.15.92:0/3336247431 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a64042300 con 0x7f9a70107400 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.030+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9a70103dc0 con 0x7f9a60038fc0 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.031+0000 7f9a6dffb640 1 -- 172.21.15.92:0/3336247431 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9a70103dc0 con 0x7f9a60038fc0 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9a60038fc0 msgr2=0x7f9a6003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9a60038fc0 0x7f9a6003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0098c0 tx=0x7f9a5c006db0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 msgr2=0x7f9a7007ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a7007ca00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a640097f0 tx=0x7f9a64030d00 comp rx=0 tx=0).stop 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 shutdown_connections 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9a60038fc0 0x7f9a6003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 --2- 172.21.15.92:0/3336247431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70107400 0x7f9a7007ca00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 >> 172.21.15.92:0/3336247431 conn(0x7f9a70102bd0 msgr2=0x7f9a701036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 shutdown_connections 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.033+0000 7f9a76abd640 1 -- 172.21.15.92:0/3336247431 wait complete. 2024-09-11T22:04:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:Generating ssh key... 2024-09-11T22:04:57.180 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-mon[31173]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:04:57.488 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: Generating public/private rsa key pair. 2024-09-11T22:04:57.488 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: Your identification has been saved in /tmp/tmp3h19zicm/key 2024-09-11T22:04:57.488 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: Your public key has been saved in /tmp/tmp3h19zicm/key.pub 2024-09-11T22:04:57.488 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: The key fingerprint is: 2024-09-11T22:04:57.488 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: SHA256:ybVynEukp5eH0yVN4g+ry+OVLo6oV81LR36u9HB2Zjg ceph-a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: The key's randomart image is: 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: +---[RSA 3072]----+ 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | o . . | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | . * +.+ | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | SoBo+ o | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | .*+=oB.. | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | ...*oOoE +| 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | .. ++B =.= | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: | .o. .o*+o.. | 2024-09-11T22:04:57.489 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:04:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: +----[SHA256]-----+ 2024-09-11T22:04:57.798 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.999+0000 7f795aaf2640 1 Processor -- start 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.999+0000 7f795aaf2640 1 -- start start 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.999+0000 7f795aaf2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795410a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:56.999+0000 7f795aaf2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f79540757a0 con 0x7f7954107f70 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.000+0000 7f795888e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795410a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.000+0000 7f795888e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795410a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39032/0 (socket says 172.21.15.92:39032) 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.000+0000 7f795888e640 1 -- 172.21.15.92:0/1322757441 learned_addr learned my addr 172.21.15.92:0/1322757441 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.000+0000 7f795888e640 1 -- 172.21.15.92:0/1322757441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795410a8a0 con 0x7f7954107f70 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.001+0000 7f795888e640 1 --2- 172.21.15.92:0/1322757441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795410a360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e80 tx=0x7f794c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352b8d97280bec7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.001+0000 7f79537fe640 1 -- 172.21.15.92:0/1322757441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c036070 con 0x7f7954107f70 2024-09-11T22:04:57.799 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.001+0000 7f79537fe640 1 -- 172.21.15.92:0/1322757441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f794c031890 con 0x7f7954107f70 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.001+0000 7f79537fe640 1 -- 172.21.15.92:0/1322757441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c007980 con 0x7f7954107f70 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.001+0000 7f795aaf2640 1 -- 172.21.15.92:0/1322757441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 msgr2=0x7f795410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.001+0000 7f795aaf2640 1 --2- 172.21.15.92:0/1322757441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795410a360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e80 tx=0x7f794c02a7a0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 -- 172.21.15.92:0/1322757441 shutdown_connections 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 --2- 172.21.15.92:0/1322757441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795410a360 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 -- 172.21.15.92:0/1322757441 >> 172.21.15.92:0/1322757441 conn(0x7f7954101a90 msgr2=0x7f7954103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 -- 172.21.15.92:0/1322757441 shutdown_connections 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 -- 172.21.15.92:0/1322757441 wait complete. 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 Processor -- start 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 -- start start 2024-09-11T22:04:57.800 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795aaf2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f795410b7c0 con 0x7f7954107f70 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795888e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795888e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39044/0 (socket says 172.21.15.92:39044) 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.002+0000 7f795888e640 1 -- 172.21.15.92:0/2417780304 learned_addr learned my addr 172.21.15.92:0/2417780304 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.003+0000 7f795888e640 1 -- 172.21.15.92:0/2417780304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795419b8b0 con 0x7f7954107f70 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.003+0000 7f795888e640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795419b370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f794c009680 tx=0x7f794c031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.003+0000 7f7951ffb640 1 -- 172.21.15.92:0/2417780304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c041020 con 0x7f7954107f70 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.003+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795419bb40 con 0x7f7954107f70 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.003+0000 7f7951ffb640 1 -- 172.21.15.92:0/2417780304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f794c03c900 con 0x7f7954107f70 2024-09-11T22:04:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.003+0000 7f7951ffb640 1 -- 172.21.15.92:0/2417780304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c036040 con 0x7f7954107f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.003+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795419e830 con 0x7f7954107f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.004+0000 7f7951ffb640 1 -- 172.21.15.92:0/2417780304 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f794c044020 con 0x7f7954107f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.004+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79541083d0 con 0x7f7954107f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.004+0000 7f7951ffb640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7940038f70 0x7f794003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.004+0000 7f7951ffb640 1 -- 172.21.15.92:0/2417780304 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f794c06d4d0 con 0x7f7954107f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.004+0000 7f7953fff640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7940038f70 0x7f794003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.005+0000 7f7953fff640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7940038f70 0x7f794003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79440098c0 tx=0x7f7944006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.009+0000 7f7951ffb640 1 -- 172.21.15.92:0/2417780304 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f794c04b020 con 0x7f7954107f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.178+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7954102cd0 con 0x7f7940038f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.241+0000 7f7951ffb640 1 -- 172.21.15.92:0/2417780304 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7954102cd0 con 0x7f7940038f70 2024-09-11T22:04:57.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7940038f70 msgr2=0x7f794003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7940038f70 0x7f794003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79440098c0 tx=0x7f7944006db0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 msgr2=0x7f795419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795419b370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f794c009680 tx=0x7f794c031c70 comp rx=0 tx=0).stop 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 shutdown_connections 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f7940038f70 0x7f794003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 --2- 172.21.15.92:0/2417780304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7954107f70 0x7f795419b370 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.243+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 >> 172.21.15.92:0/2417780304 conn(0x7f7954101a90 msgr2=0x7f79541025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.244+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 shutdown_connections 2024-09-11T22:04:57.803 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:57.244+0000 7f795aaf2640 1 -- 172.21.15.92:0/2417780304 wait complete. 2024-09-11T22:04:58.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:58 smithi092 ceph-mon[31173]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:04:58.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:58 smithi092 ceph-mon[31173]: Generating ssh key... 2024-09-11T22:04:58.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:58 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:04:58.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:58 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:04:58.995 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPZOjNyZK2EuKPwdf02uZ7dCcZZsxYK2dHGmlkAdEfA96S6ToHQf/4wwzjjAWKHZyKwTjVLe5mR0eY3OaXKmtrLmNq8idjM+FHYGJe6l692QhKKxjp1fo3DS/kLdw+cT95PVlgC6Rd43C2EwWWta4vKW+znQlXRCVsrmMQXrL09obYX763gJHP7zFCSYWnYjpXiGeyn032zFHeo63K84bpONl1z6s71uC3RUOShldhnPJWCJWHQBXl8xv8DtfnhFLhCPn37p3U2DZ5JNJ0vRxeJ6uHadJ/xoxm2U/euS0dmmt4myzV20GBB7KvdVeYK10iTKnFtgQKEKorvrPmSlVwjrFH/vNCAg8cmaFv6GzyldP9QB1yw9+2KtgWguwseTaNKVeHY2+Vea41J3sn4k2rKy06glPfFVCAV+GLTTmj3nU3v8boZtAfj1ZqPAAWjhbx24qfwZyb4IVKfYKe0aT+ADVXw9D8ckdewUFrV4PFIP6RJAvEbASzlV3VtTmarcs= ceph-a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:04:58.995 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.188+0000 7f924eb01640 1 Processor -- start 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.188+0000 7f924eb01640 1 -- start start 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.188+0000 7f924eb01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f9248107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.188+0000 7f924eb01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9248107df0 con 0x7f9248107420 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.188+0000 7f924c89d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f9248107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.188+0000 7f924c89d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f9248107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39046/0 (socket says 172.21.15.92:39046) 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.188+0000 7f924c89d640 1 -- 172.21.15.92:0/685226858 learned_addr learned my addr 172.21.15.92:0/685226858 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.189+0000 7f924c89d640 1 -- 172.21.15.92:0/685226858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9248108620 con 0x7f9248107420 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.189+0000 7f924c89d640 1 --2- 172.21.15.92:0/685226858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f9248107820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f923c009820 tx=0x7f923c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509788ded6d906ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f92477fe640 1 -- 172.21.15.92:0/685226858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f923c037070 con 0x7f9248107420 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f92477fe640 1 -- 172.21.15.92:0/685226858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f923c0328d0 con 0x7f9248107420 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f92477fe640 1 -- 172.21.15.92:0/685226858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f923c0307d0 con 0x7f9248107420 2024-09-11T22:04:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f924eb01640 1 -- 172.21.15.92:0/685226858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 msgr2=0x7f9248107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f924eb01640 1 --2- 172.21.15.92:0/685226858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f9248107820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f923c009820 tx=0x7f923c02a270 comp rx=0 tx=0).stop 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f924eb01640 1 -- 172.21.15.92:0/685226858 shutdown_connections 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f924eb01640 1 --2- 172.21.15.92:0/685226858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f9248107820 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f924eb01640 1 -- 172.21.15.92:0/685226858 >> 172.21.15.92:0/685226858 conn(0x7f9248102bb0 msgr2=0x7f9248104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f924eb01640 1 -- 172.21.15.92:0/685226858 shutdown_connections 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.190+0000 7f924eb01640 1 -- 172.21.15.92:0/685226858 wait complete. 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924eb01640 1 Processor -- start 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924eb01640 1 -- start start 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924eb01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f924819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924eb01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f924810b9d0 con 0x7f9248107420 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924c89d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f924819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924c89d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f924819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39050/0 (socket says 172.21.15.92:39050) 2024-09-11T22:04:58.997 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924c89d640 1 -- 172.21.15.92:0/2132260349 learned_addr learned my addr 172.21.15.92:0/2132260349 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924c89d640 1 -- 172.21.15.92:0/2132260349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924819b8d0 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.191+0000 7f924c89d640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f924819b390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f923c0097f0 tx=0x7f923c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.192+0000 7f9245ffb640 1 -- 172.21.15.92:0/2132260349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f923c037040 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.192+0000 7f9245ffb640 1 -- 172.21.15.92:0/2132260349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f923c032bb0 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.192+0000 7f9245ffb640 1 -- 172.21.15.92:0/2132260349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f923c03a390 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.192+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924819bb60 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.192+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924819e850 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.193+0000 7f9245ffb640 1 -- 172.21.15.92:0/2132260349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f923c03b3f0 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.193+0000 7f9245ffb640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9238038fc0 0x7f923803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.193+0000 7f9245ffb640 1 -- 172.21.15.92:0/2132260349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f923c06c190 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.193+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924810bb50 con 0x7f9248107420 2024-09-11T22:04:58.998 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.193+0000 7f9247fff640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9238038fc0 0x7f923803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.193+0000 7f9247fff640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9238038fc0 0x7f923803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92340098c0 tx=0x7f9234006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.198+0000 7f9245ffb640 1 -- 172.21.15.92:0/2132260349 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f923c032d60 con 0x7f9248107420 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.368+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f9248103db0 con 0x7f9238038fc0 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.368+0000 7f9245ffb640 1 -- 172.21.15.92:0/2132260349 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f9248103db0 con 0x7f9238038fc0 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9238038fc0 msgr2=0x7f923803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9238038fc0 0x7f923803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92340098c0 tx=0x7f9234006db0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 msgr2=0x7f924819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f924819b390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f923c0097f0 tx=0x7f923c030d00 comp rx=0 tx=0).stop 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 shutdown_connections 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f9238038fc0 0x7f923803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 --2- 172.21.15.92:0/2132260349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9248107420 0x7f924819b390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 >> 172.21.15.92:0/2132260349 conn(0x7f9248102bb0 msgr2=0x7f92481036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 shutdown_connections 2024-09-11T22:04:58.999 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:58.371+0000 7f924eb01640 1 -- 172.21.15.92:0/2132260349 wait complete. 2024-09-11T22:04:59.000 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:04:59.000 INFO:teuthology.orchestra.run.smithi092.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T22:04:59.000 INFO:teuthology.orchestra.run.smithi092.stdout:Adding host smithi092... 2024-09-11T22:04:59.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:04:59 smithi092 ceph-mon[31173]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:00.365 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:00 smithi092 ceph-mon[31173]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:01.328 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:01 smithi092 ceph-mon[31173]: Deploying cephadm binary to smithi092 2024-09-11T22:05:03.255 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Added host 'smithi092' with addr '172.21.15.92' 2024-09-11T22:05:03.255 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.412+0000 7fa6ea7aa640 1 Processor -- start 2024-09-11T22:05:03.255 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.412+0000 7fa6ea7aa640 1 -- start start 2024-09-11T22:05:03.255 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.412+0000 7fa6ea7aa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e407c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.412+0000 7fa6ea7aa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e407c850 con 0x7fa6e407de20 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.412+0000 7fa6e3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e407c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.412+0000 7fa6e3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e407c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39060/0 (socket says 172.21.15.92:39060) 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.412+0000 7fa6e3fff640 1 -- 172.21.15.92:0/3884882174 learned_addr learned my addr 172.21.15.92:0/3884882174 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.413+0000 7fa6e3fff640 1 -- 172.21.15.92:0/3884882174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e407c9d0 con 0x7fa6e407de20 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.413+0000 7fa6e3fff640 1 --2- 172.21.15.92:0/3884882174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e407c280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009820 tx=0x7fa6d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9187d1fa958de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.413+0000 7fa6e37fe640 1 -- 172.21.15.92:0/3884882174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d4037070 con 0x7fa6e407de20 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.413+0000 7fa6e37fe640 1 -- 172.21.15.92:0/3884882174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa6d40328d0 con 0x7fa6e407de20 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.413+0000 7fa6e37fe640 1 -- 172.21.15.92:0/3884882174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d40307d0 con 0x7fa6e407de20 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.414+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/3884882174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 msgr2=0x7fa6e407c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.414+0000 7fa6ea7aa640 1 --2- 172.21.15.92:0/3884882174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e407c280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009820 tx=0x7fa6d402a270 comp rx=0 tx=0).stop 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.414+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/3884882174 shutdown_connections 2024-09-11T22:05:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.414+0000 7fa6ea7aa640 1 --2- 172.21.15.92:0/3884882174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e407c280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.414+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/3884882174 >> 172.21.15.92:0/3884882174 conn(0x7fa6e4102bb0 msgr2=0x7fa6e4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.414+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/3884882174 shutdown_connections 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.414+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/3884882174 wait complete. 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6ea7aa640 1 Processor -- start 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6ea7aa640 1 -- start start 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6ea7aa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6ea7aa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e410d9f0 con 0x7fa6e407de20 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6e3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6e3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e419d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39068/0 (socket says 172.21.15.92:39068) 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6e3fff640 1 -- 172.21.15.92:0/1043191469 learned_addr learned my addr 172.21.15.92:0/1043191469 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6e3fff640 1 -- 172.21.15.92:0/1043191469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e419da90 con 0x7fa6e407de20 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.415+0000 7fa6e3fff640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e419d550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4032e10 tx=0x7fa6d4032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.416+0000 7fa6e1ffb640 1 -- 172.21.15.92:0/1043191469 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d4042070 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.416+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e419dd20 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.416+0000 7fa6e1ffb640 1 -- 172.21.15.92:0/1043191469 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa6d4030c80 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.416+0000 7fa6e1ffb640 1 -- 172.21.15.92:0/1043191469 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d4037040 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.416+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e41a0a10 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.416+0000 7fa6e1ffb640 1 -- 172.21.15.92:0/1043191469 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa6d4045070 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.416+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e407c300 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.417+0000 7fa6e1ffb640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fa6d0038f70 0x7fa6d003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.417+0000 7fa6e1ffb640 1 -- 172.21.15.92:0/1043191469 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6d406bf30 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.417+0000 7fa6dbfff640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fa6d0038f70 0x7fa6d003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.417+0000 7fa6dbfff640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fa6d0038f70 0x7fa6d003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0098c0 tx=0x7fa6cc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.421+0000 7fa6e1ffb640 1 -- 172.21.15.92:0/1043191469 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6d4045360 con 0x7fa6e407de20 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:04:59.594+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}) -- 0x7fa6e4103e40 con 0x7fa6d0038f70 2024-09-11T22:05:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.761+0000 7fa6e1ffb640 1 -- 172.21.15.92:0/1043191469 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa6e4103e40 con 0x7fa6d0038f70 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fa6d0038f70 msgr2=0x7fa6d003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fa6d0038f70 0x7fa6d003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0098c0 tx=0x7fa6cc006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 msgr2=0x7fa6e419d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e419d550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4032e10 tx=0x7fa6d4032e40 comp rx=0 tx=0).stop 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 shutdown_connections 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fa6d0038f70 0x7fa6d003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 --2- 172.21.15.92:0/1043191469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e407de20 0x7fa6e419d550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 >> 172.21.15.92:0/1043191469 conn(0x7fa6e4102bb0 msgr2=0x7fa6e4103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 shutdown_connections 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:02.764+0000 7fa6ea7aa640 1 -- 172.21.15.92:0/1043191469 wait complete. 2024-09-11T22:05:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mon service... 2024-09-11T22:05:04.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:03 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:05:04.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:03 smithi092 ceph-mon[31173]: Added host smithi092 2024-09-11T22:05:04.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:03 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.648+0000 7f335b5b2640 1 Processor -- start 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.648+0000 7f335b5b2640 1 -- start start 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.649+0000 7f335b5b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335410a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.649+0000 7f335b5b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3354075820 con 0x7f3354107ef0 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.649+0000 7f335934e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335410a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.649+0000 7f335934e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335410a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39074/0 (socket says 172.21.15.92:39074) 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.649+0000 7f335934e640 1 -- 172.21.15.92:0/3086014521 learned_addr learned my addr 172.21.15.92:0/3086014521 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.649+0000 7f335934e640 1 -- 172.21.15.92:0/3086014521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335410a820 con 0x7f3354107ef0 2024-09-11T22:05:04.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.650+0000 7f335934e640 1 --2- 172.21.15.92:0/3086014521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335410a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3344009820 tx=0x7f334402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4a95ceec389b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:04.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.650+0000 7f334bfff640 1 -- 172.21.15.92:0/3086014521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3344037070 con 0x7f3354107ef0 2024-09-11T22:05:04.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.650+0000 7f334bfff640 1 -- 172.21.15.92:0/3086014521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f33440328d0 con 0x7f3354107ef0 2024-09-11T22:05:04.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.650+0000 7f334bfff640 1 -- 172.21.15.92:0/3086014521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33440307d0 con 0x7f3354107ef0 2024-09-11T22:05:04.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 -- 172.21.15.92:0/3086014521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 msgr2=0x7f335410a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 --2- 172.21.15.92:0/3086014521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335410a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3344009820 tx=0x7f334402a270 comp rx=0 tx=0).stop 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 -- 172.21.15.92:0/3086014521 shutdown_connections 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 --2- 172.21.15.92:0/3086014521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335410a2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 -- 172.21.15.92:0/3086014521 >> 172.21.15.92:0/3086014521 conn(0x7f3354101a90 msgr2=0x7f3354103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 -- 172.21.15.92:0/3086014521 shutdown_connections 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 -- 172.21.15.92:0/3086014521 wait complete. 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 Processor -- start 2024-09-11T22:05:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 -- start start 2024-09-11T22:05:04.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:04.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.651+0000 7f335b5b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f335410b800 con 0x7f3354107ef0 2024-09-11T22:05:04.661 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f335934e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:04.661 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f335934e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39090/0 (socket says 172.21.15.92:39090) 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f335934e640 1 -- 172.21.15.92:0/2094870548 learned_addr learned my addr 172.21.15.92:0/2094870548 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f335934e640 1 -- 172.21.15.92:0/2094870548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335419b8a0 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f335934e640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335419b360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3344032e10 tx=0x7f3344032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f334a7fc640 1 -- 172.21.15.92:0/2094870548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3344042070 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f334a7fc640 1 -- 172.21.15.92:0/2094870548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3344030c80 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335419bb30 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f334a7fc640 1 -- 172.21.15.92:0/2094870548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3344037040 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.652+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335419e820 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.653+0000 7f334a7fc640 1 -- 172.21.15.92:0/2094870548 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3344045070 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.653+0000 7f334a7fc640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f3340038fc0 0x7f334003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.653+0000 7f334a7fc640 1 -- 172.21.15.92:0/2094870548 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f334406beb0 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.653+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3354108350 con 0x7f3354107ef0 2024-09-11T22:05:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.653+0000 7f3358b4d640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f3340038fc0 0x7f334003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.654+0000 7f3358b4d640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f3340038fc0 0x7f334003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f33500098c0 tx=0x7f3350006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.658+0000 7f334a7fc640 1 -- 172.21.15.92:0/2094870548 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f334404b170 con 0x7f3354107ef0 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.845+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3354102ce0 con 0x7f3340038fc0 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.849+0000 7f334a7fc640 1 -- 172.21.15.92:0/2094870548 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3354102ce0 con 0x7f3340038fc0 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f3340038fc0 msgr2=0x7f334003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f3340038fc0 0x7f334003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f33500098c0 tx=0x7f3350006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 msgr2=0x7f335419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335419b360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3344032e10 tx=0x7f3344032e40 comp rx=0 tx=0).stop 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 shutdown_connections 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f3340038fc0 0x7f334003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 --2- 172.21.15.92:0/2094870548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3354107ef0 0x7f335419b360 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 >> 172.21.15.92:0/2094870548 conn(0x7f3354101a90 msgr2=0x7f33541025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 shutdown_connections 2024-09-11T22:05:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:03.852+0000 7f335b5b2640 1 -- 172.21.15.92:0/2094870548 wait complete. 2024-09-11T22:05:04.664 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mgr service... 2024-09-11T22:05:04.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:04 smithi092 ceph-mon[31173]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:04.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:04 smithi092 ceph-mon[31173]: Saving service mon spec with placement count:5 2024-09-11T22:05:04.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:04 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:05:06.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:06 smithi092 ceph-mon[31173]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:06.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:06 smithi092 ceph-mon[31173]: Saving service mgr spec with placement count:2 2024-09-11T22:05:06.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:06 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:05:06.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:06 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:05:06.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:06 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:05:06.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T22:05:06.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.057+0000 7f006fb78640 1 Processor -- start 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.057+0000 7f006fb78640 1 -- start start 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.057+0000 7f006fb78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f0068106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.057+0000 7f006fb78640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0068106ce0 con 0x7f0068106310 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.057+0000 7f006d914640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f0068106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.057+0000 7f006d914640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f0068106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39098/0 (socket says 172.21.15.92:39098) 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.057+0000 7f006d914640 1 -- 172.21.15.92:0/1124746987 learned_addr learned my addr 172.21.15.92:0/1124746987 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.058+0000 7f006d914640 1 -- 172.21.15.92:0/1124746987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0068107510 con 0x7f0068106310 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.058+0000 7f006d914640 1 --2- 172.21.15.92:0/1124746987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f0068106710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0064009820 tx=0x7f006402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a09d0170c534e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.058+0000 7f006c912640 1 -- 172.21.15.92:0/1124746987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0064037070 con 0x7f0068106310 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006c912640 1 -- 172.21.15.92:0/1124746987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f00640328d0 con 0x7f0068106310 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006c912640 1 -- 172.21.15.92:0/1124746987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00640307d0 con 0x7f0068106310 2024-09-11T22:05:06.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006fb78640 1 -- 172.21.15.92:0/1124746987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 msgr2=0x7f0068106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006fb78640 1 --2- 172.21.15.92:0/1124746987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f0068106710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0064009820 tx=0x7f006402a270 comp rx=0 tx=0).stop 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006fb78640 1 -- 172.21.15.92:0/1124746987 shutdown_connections 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006fb78640 1 --2- 172.21.15.92:0/1124746987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f0068106710 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006fb78640 1 -- 172.21.15.92:0/1124746987 >> 172.21.15.92:0/1124746987 conn(0x7f0068101ae0 msgr2=0x7f0068103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006fb78640 1 -- 172.21.15.92:0/1124746987 shutdown_connections 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.059+0000 7f006fb78640 1 -- 172.21.15.92:0/1124746987 wait complete. 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006fb78640 1 Processor -- start 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006fb78640 1 -- start start 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006fb78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f006819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006fb78640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f006810a8c0 con 0x7f0068106310 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006d914640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f006819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006d914640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f006819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39108/0 (socket says 172.21.15.92:39108) 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006d914640 1 -- 172.21.15.92:0/2123468779 learned_addr learned my addr 172.21.15.92:0/2123468779 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006d914640 1 -- 172.21.15.92:0/2123468779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006819a7c0 con 0x7f0068106310 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f006d914640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f006819a280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0064009940 tx=0x7f0064032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.060+0000 7f005effd640 1 -- 172.21.15.92:0/2123468779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0064042070 con 0x7f0068106310 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.061+0000 7f005effd640 1 -- 172.21.15.92:0/2123468779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0064030c80 con 0x7f0068106310 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.061+0000 7f005effd640 1 -- 172.21.15.92:0/2123468779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0064037040 con 0x7f0068106310 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.061+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006819aa50 con 0x7f0068106310 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.061+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006819d740 con 0x7f0068106310 2024-09-11T22:05:06.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.061+0000 7f005effd640 1 -- 172.21.15.92:0/2123468779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0064045070 con 0x7f0068106310 2024-09-11T22:05:06.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.061+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006810aa40 con 0x7f0068106310 2024-09-11T22:05:06.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.061+0000 7f005effd640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f0054038fc0 0x7f005403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.062+0000 7f005effd640 1 -- 172.21.15.92:0/2123468779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f006406be70 con 0x7f0068106310 2024-09-11T22:05:06.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.062+0000 7f006d113640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f0054038fc0 0x7f005403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.062+0000 7f006d113640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f0054038fc0 0x7f005403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f00580098c0 tx=0x7f0058006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.555 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.066+0000 7f005effd640 1 -- 172.21.15.92:0/2123468779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0064045360 con 0x7f0068106310 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.255+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0068102ca0 con 0x7f0054038fc0 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.259+0000 7f005effd640 1 -- 172.21.15.92:0/2123468779 <== mgr.14118 v2:172.21.15.92:6800/733069068 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0068102ca0 con 0x7f0054038fc0 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f0054038fc0 msgr2=0x7f005403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f0054038fc0 0x7f005403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f00580098c0 tx=0x7f0058006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 msgr2=0x7f006819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f006819a280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0064009940 tx=0x7f0064032e10 comp rx=0 tx=0).stop 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 shutdown_connections 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f0054038fc0 0x7f005403b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 --2- 172.21.15.92:0/2123468779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0068106310 0x7f006819a280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 >> 172.21.15.92:0/2123468779 conn(0x7f0068101ae0 msgr2=0x7f0068102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 shutdown_connections 2024-09-11T22:05:06.556 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:05.261+0000 7f006fb78640 1 -- 172.21.15.92:0/2123468779 wait complete. 2024-09-11T22:05:08.305 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.976+0000 7fcad6506640 1 Processor -- start 2024-09-11T22:05:08.305 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.977+0000 7fcad6506640 1 -- start start 2024-09-11T22:05:08.305 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.977+0000 7fcad6506640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad010a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:08.305 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.977+0000 7fcad6506640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcad0075820 con 0x7fcad0107ef0 2024-09-11T22:05:08.305 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.977+0000 7fcacffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad010a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:08.306 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.977+0000 7fcacffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad010a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52140/0 (socket says 172.21.15.92:52140) 2024-09-11T22:05:08.306 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.977+0000 7fcacffff640 1 -- 172.21.15.92:0/2971433181 learned_addr learned my addr 172.21.15.92:0/2971433181 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:08.306 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.977+0000 7fcacffff640 1 -- 172.21.15.92:0/2971433181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad010a820 con 0x7fcad0107ef0 2024-09-11T22:05:08.306 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.978+0000 7fcacffff640 1 --2- 172.21.15.92:0/2971433181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad010a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009820 tx=0x7fcac002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb33031d375ad2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.978+0000 7fcaceffd640 1 -- 172.21.15.92:0/2971433181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0037070 con 0x7fcad0107ef0 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.978+0000 7fcaceffd640 1 -- 172.21.15.92:0/2971433181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac00328f0 con 0x7fcad0107ef0 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.978+0000 7fcaceffd640 1 -- 172.21.15.92:0/2971433181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0030740 con 0x7fcad0107ef0 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.979+0000 7fcad6506640 1 -- 172.21.15.92:0/2971433181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 msgr2=0x7fcad010a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.979+0000 7fcad6506640 1 --2- 172.21.15.92:0/2971433181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad010a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009820 tx=0x7fcac002a270 comp rx=0 tx=0).stop 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.979+0000 7fcad6506640 1 -- 172.21.15.92:0/2971433181 shutdown_connections 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.979+0000 7fcad6506640 1 --2- 172.21.15.92:0/2971433181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad010a2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.979+0000 7fcad6506640 1 -- 172.21.15.92:0/2971433181 >> 172.21.15.92:0/2971433181 conn(0x7fcad0101a90 msgr2=0x7fcad0103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.979+0000 7fcad6506640 1 -- 172.21.15.92:0/2971433181 shutdown_connections 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.979+0000 7fcad6506640 1 -- 172.21.15.92:0/2971433181 wait complete. 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.980+0000 7fcad6506640 1 Processor -- start 2024-09-11T22:05:08.307 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.980+0000 7fcad6506640 1 -- start start 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.980+0000 7fcad6506640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.980+0000 7fcad6506640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcad010b840 con 0x7fcad0107ef0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.980+0000 7fcacffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.980+0000 7fcacffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52144/0 (socket says 172.21.15.92:52144) 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.980+0000 7fcacffff640 1 -- 172.21.15.92:0/1763685869 learned_addr learned my addr 172.21.15.92:0/1763685869 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.981+0000 7fcacffff640 1 -- 172.21.15.92:0/1763685869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad019b8c0 con 0x7fcad0107ef0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.981+0000 7fcacffff640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad019b380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097f0 tx=0x7fcac0030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.981+0000 7fcacd7fa640 1 -- 172.21.15.92:0/1763685869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0042070 con 0x7fcad0107ef0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.981+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad019bb50 con 0x7fcad0107ef0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.981+0000 7fcacd7fa640 1 -- 172.21.15.92:0/1763685869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac0030d80 con 0x7fcad0107ef0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.981+0000 7fcacd7fa640 1 -- 172.21.15.92:0/1763685869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0037040 con 0x7fcad0107ef0 2024-09-11T22:05:08.308 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.981+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad019e840 con 0x7fcad0107ef0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.982+0000 7fcacd7fa640 1 -- 172.21.15.92:0/1763685869 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcac0045070 con 0x7fcad0107ef0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.982+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad0108350 con 0x7fcad0107ef0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.982+0000 7fcacd7fa640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fcab8038f70 0x7fcab803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.982+0000 7fcacd7fa640 1 -- 172.21.15.92:0/1763685869 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcac006c080 con 0x7fcad0107ef0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.983+0000 7fcacf7fe640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fcab8038f70 0x7fcab803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.983+0000 7fcacf7fe640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fcab8038f70 0x7fcab803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcac40098c0 tx=0x7fcac4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:06.987+0000 7fcacd7fa640 1 -- 172.21.15.92:0/1763685869 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcac0045520 con 0x7fcad0107ef0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.165+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fcad019ee40 con 0x7fcad0107ef0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.168+0000 7fcacd7fa640 1 -- 172.21.15.92:0/1763685869 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fcac003add0 con 0x7fcad0107ef0 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.170+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fcab8038f70 msgr2=0x7fcab803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.170+0000 7fcad6506640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fcab8038f70 0x7fcab803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcac40098c0 tx=0x7fcac4006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.170+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 msgr2=0x7fcad019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:08.309 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.170+0000 7fcad6506640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad019b380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097f0 tx=0x7fcac0030c70 comp rx=0 tx=0).stop 2024-09-11T22:05:08.310 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.171+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 shutdown_connections 2024-09-11T22:05:08.310 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.171+0000 7fcad6506640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fcab8038f70 0x7fcab803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:08.310 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.171+0000 7fcad6506640 1 --2- 172.21.15.92:0/1763685869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcad0107ef0 0x7fcad019b380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:08.310 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.171+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 >> 172.21.15.92:0/1763685869 conn(0x7fcad0101a90 msgr2=0x7fcad01025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:08.310 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.171+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 shutdown_connections 2024-09-11T22:05:08.310 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:07.171+0000 7fcad6506640 1 -- 172.21.15.92:0/1763685869 wait complete. 2024-09-11T22:05:08.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:08 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1763685869' entity='client.admin' 2024-09-11T22:05:08.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:08 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:05:09.393 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.692+0000 7fea8fdf1640 1 Processor -- start 2024-09-11T22:05:09.393 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.693+0000 7fea8fdf1640 1 -- start start 2024-09-11T22:05:09.393 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.693+0000 7fea8fdf1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.393 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.693+0000 7fea8fdf1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea8807c870 con 0x7fea8807de40 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.693+0000 7fea8db8d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.693+0000 7fea8db8d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52154/0 (socket says 172.21.15.92:52154) 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.693+0000 7fea8db8d640 1 -- 172.21.15.92:0/1163163311 learned_addr learned my addr 172.21.15.92:0/1163163311 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.693+0000 7fea8db8d640 1 -- 172.21.15.92:0/1163163311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8807c9f0 con 0x7fea8807de40 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.694+0000 7fea8db8d640 1 --2- 172.21.15.92:0/1163163311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8807c2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea84009820 tx=0x7fea8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99f452f0874ce5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.694+0000 7fea8cb8b640 1 -- 172.21.15.92:0/1163163311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea84037070 con 0x7fea8807de40 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.694+0000 7fea8cb8b640 1 -- 172.21.15.92:0/1163163311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea840328f0 con 0x7fea8807de40 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.694+0000 7fea8cb8b640 1 -- 172.21.15.92:0/1163163311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea84030740 con 0x7fea8807de40 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.695+0000 7fea8fdf1640 1 -- 172.21.15.92:0/1163163311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 msgr2=0x7fea8807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.695+0000 7fea8fdf1640 1 --2- 172.21.15.92:0/1163163311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8807c2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea84009820 tx=0x7fea8402a270 comp rx=0 tx=0).stop 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.695+0000 7fea8fdf1640 1 -- 172.21.15.92:0/1163163311 shutdown_connections 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.695+0000 7fea8fdf1640 1 --2- 172.21.15.92:0/1163163311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8807c2a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.695+0000 7fea8fdf1640 1 -- 172.21.15.92:0/1163163311 >> 172.21.15.92:0/1163163311 conn(0x7fea88102bb0 msgr2=0x7fea88104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.695+0000 7fea8fdf1640 1 -- 172.21.15.92:0/1163163311 shutdown_connections 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.695+0000 7fea8fdf1640 1 -- 172.21.15.92:0/1163163311 wait complete. 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8fdf1640 1 Processor -- start 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8fdf1640 1 -- start start 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8fdf1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8fdf1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810da90 con 0x7fea8807de40 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8db8d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8db8d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52158/0 (socket says 172.21.15.92:52158) 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8db8d640 1 -- 172.21.15.92:0/4084817428 learned_addr learned my addr 172.21.15.92:0/4084817428 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8db8d640 1 -- 172.21.15.92:0/4084817428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819dae0 con 0x7fea8807de40 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.696+0000 7fea8db8d640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8819d5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea840097f0 tx=0x7fea84030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.697+0000 7fea7effd640 1 -- 172.21.15.92:0/4084817428 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea84042070 con 0x7fea8807de40 2024-09-11T22:05:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.697+0000 7fea7effd640 1 -- 172.21.15.92:0/4084817428 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea84030d80 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.697+0000 7fea7effd640 1 -- 172.21.15.92:0/4084817428 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea84037040 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.697+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8819dd70 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.697+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881a0a60 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.698+0000 7fea7effd640 1 -- 172.21.15.92:0/4084817428 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fea84045070 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.698+0000 7fea7effd640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fea74038f70 0x7fea7403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.698+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8807c320 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.698+0000 7fea7effd640 1 -- 172.21.15.92:0/4084817428 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea8406c2f0 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.698+0000 7fea8d38c640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fea74038f70 0x7fea7403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.699+0000 7fea8d38c640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fea74038f70 0x7fea7403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fea780098c0 tx=0x7fea78006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.706+0000 7fea7effd640 1 -- 172.21.15.92:0/4084817428 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea84045520 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.881+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fea881a0c30 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.885+0000 7fea7effd640 1 -- 172.21.15.92:0/4084817428 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fea840500c0 con 0x7fea8807de40 2024-09-11T22:05:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fea74038f70 msgr2=0x7fea7403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fea74038f70 0x7fea7403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fea780098c0 tx=0x7fea78006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 msgr2=0x7fea8819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8819d5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea840097f0 tx=0x7fea84030c70 comp rx=0 tx=0).stop 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 shutdown_connections 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fea74038f70 0x7fea7403b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 --2- 172.21.15.92:0/4084817428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8807de40 0x7fea8819d5a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 >> 172.21.15.92:0/4084817428 conn(0x7fea88102bb0 msgr2=0x7fea881036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.887+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 shutdown_connections 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:08.888+0000 7fea8fdf1640 1 -- 172.21.15.92:0/4084817428 wait complete. 2024-09-11T22:05:09.397 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling the dashboard module... 2024-09-11T22:05:09.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:09 smithi092 ceph-mon[31173]: from='mgr.14118 172.21.15.92:0/2486752492' entity='mgr.x' 2024-09-11T22:05:09.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:09 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4084817428' entity='client.admin' 2024-09-11T22:05:10.862 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:10 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1229504181' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T22:05:10.862 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: ignoring --setuser ceph since I am not root 2024-09-11T22:05:10.862 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: ignoring --setgroup ceph since I am not root 2024-09-11T22:05:11.158 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:10.879+0000 7f8ec9ae0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:05:11.158 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:10.936+0000 7f8ec9ae0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:05:11.158 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:11.085+0000 7f8ec9ae0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:05:11.159 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:11.157+0000 7f8ec9ae0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:05:11.409 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:11.304+0000 7f8ec9ae0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.790+0000 7fb5632f7640 1 Processor -- start 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.790+0000 7fb5632f7640 1 -- start start 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.791+0000 7fb5632f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.791+0000 7fb5632f7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c107df0 con 0x7fb55c107420 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.791+0000 7fb561093640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.791+0000 7fb561093640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52172/0 (socket says 172.21.15.92:52172) 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.791+0000 7fb561093640 1 -- 172.21.15.92:0/806804910 learned_addr learned my addr 172.21.15.92:0/806804910 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.791+0000 7fb561093640 1 -- 172.21.15.92:0/806804910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c108620 con 0x7fb55c107420 2024-09-11T22:05:11.415 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.792+0000 7fb561093640 1 --2- 172.21.15.92:0/806804910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c107820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009820 tx=0x7fb54c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16bbee02bd5340d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.792+0000 7fb553fff640 1 -- 172.21.15.92:0/806804910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb54c037070 con 0x7fb55c107420 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.792+0000 7fb553fff640 1 -- 172.21.15.92:0/806804910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb54c0328f0 con 0x7fb55c107420 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.792+0000 7fb553fff640 1 -- 172.21.15.92:0/806804910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb54c030740 con 0x7fb55c107420 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.792+0000 7fb5632f7640 1 -- 172.21.15.92:0/806804910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 msgr2=0x7fb55c107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.792+0000 7fb5632f7640 1 --2- 172.21.15.92:0/806804910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c107820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009820 tx=0x7fb54c02a270 comp rx=0 tx=0).stop 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 -- 172.21.15.92:0/806804910 shutdown_connections 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 --2- 172.21.15.92:0/806804910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c107820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 -- 172.21.15.92:0/806804910 >> 172.21.15.92:0/806804910 conn(0x7fb55c102bb0 msgr2=0x7fb55c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 -- 172.21.15.92:0/806804910 shutdown_connections 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 -- 172.21.15.92:0/806804910 wait complete. 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 Processor -- start 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 -- start start 2024-09-11T22:05:11.416 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb5632f7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c10b9d0 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb561093640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb561093640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52182/0 (socket says 172.21.15.92:52182) 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.793+0000 7fb561093640 1 -- 172.21.15.92:0/1229504181 learned_addr learned my addr 172.21.15.92:0/1229504181 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.794+0000 7fb561093640 1 -- 172.21.15.92:0/1229504181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c19b920 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.794+0000 7fb561093640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c19b3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009940 tx=0x7fb54c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.794+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb54c042070 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.794+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb54c030e30 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.794+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55c19bbb0 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.794+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb54c037040 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.794+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55c19e8a0 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.795+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb54c045070 con 0x7fb55c107420 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.795+0000 7fb5527fc640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fb548038f70 0x7fb54803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:11.417 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.795+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb54c06c0c0 con 0x7fb55c107420 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.795+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55c10bb50 con 0x7fb55c107420 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.795+0000 7fb560892640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fb548038f70 0x7fb54803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.795+0000 7fb560892640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fb548038f70 0x7fb54803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5580098c0 tx=0x7fb558006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:09.800+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb54c045360 con 0x7fb55c107420 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.020+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb55c19ea70 con 0x7fb55c107420 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.730+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb54c06ba50 con 0x7fb55c107420 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.732+0000 7fb5527fc640 1 -- 172.21.15.92:0/1229504181 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb54c03b330 con 0x7fb55c107420 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fb548038f70 msgr2=0x7fb54803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fb548038f70 0x7fb54803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5580098c0 tx=0x7fb558006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 msgr2=0x7fb55c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c19b3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009940 tx=0x7fb54c030cc0 comp rx=0 tx=0).stop 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 shutdown_connections 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fb548038f70 0x7fb54803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:11.418 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 --2- 172.21.15.92:0/1229504181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb55c107420 0x7fb55c19b3e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:11.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 >> 172.21.15.92:0/1229504181 conn(0x7fb55c102bb0 msgr2=0x7fb55c1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:11.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 shutdown_connections 2024-09-11T22:05:11.419 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:10.733+0000 7fb5632f7640 1 -- 172.21.15.92:0/1229504181 wait complete. 2024-09-11T22:05:11.718 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:11.453+0000 7f8ec9ae0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:05:12.003 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1229504181' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T22:05:12.003 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-mon[31173]: mgrmap e9: x(active, since 17s) 2024-09-11T22:05:12.003 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:11.774+0000 7f8ec9ae0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:05:12.003 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:11.839+0000 7f8ec9ae0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:05:12.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:12.029+0000 7f8ec9ae0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:05:12.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:05:12.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T22:05:12.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:05:12.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T22:05:12.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T22:05:12.563 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f1400c3e640 1 Processor -- start 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f1400c3e640 1 -- start start 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f1400c3e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc10a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f1400c3e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc07be10 con 0x7f13fc107ec0 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f13fa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc10a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f13fa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc10a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52218/0 (socket says 172.21.15.92:52218) 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f13fa59c640 1 -- 172.21.15.92:0/3529532586 learned_addr learned my addr 172.21.15.92:0/3529532586 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.796+0000 7f13fa59c640 1 -- 172.21.15.92:0/3529532586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc10a7f0 con 0x7f13fc107ec0 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.797+0000 7f13fa59c640 1 --2- 172.21.15.92:0/3529532586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc10a2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009820 tx=0x7f13ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc500bcb652f9b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.797+0000 7f13f9d9b640 1 -- 172.21.15.92:0/3529532586 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec037070 con 0x7f13fc107ec0 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.797+0000 7f13f9d9b640 1 -- 172.21.15.92:0/3529532586 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13ec0328f0 con 0x7f13fc107ec0 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.797+0000 7f13f9d9b640 1 -- 172.21.15.92:0/3529532586 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec030740 con 0x7f13fc107ec0 2024-09-11T22:05:12.564 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.798+0000 7f1400c3e640 1 -- 172.21.15.92:0/3529532586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 msgr2=0x7f13fc10a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.798+0000 7f1400c3e640 1 --2- 172.21.15.92:0/3529532586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc10a2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009820 tx=0x7f13ec02a270 comp rx=0 tx=0).stop 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.798+0000 7f1400c3e640 1 -- 172.21.15.92:0/3529532586 shutdown_connections 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.798+0000 7f1400c3e640 1 --2- 172.21.15.92:0/3529532586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc10a2b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.798+0000 7f1400c3e640 1 -- 172.21.15.92:0/3529532586 >> 172.21.15.92:0/3529532586 conn(0x7f13fc101ac0 msgr2=0x7f13fc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.798+0000 7f1400c3e640 1 -- 172.21.15.92:0/3529532586 shutdown_connections 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.798+0000 7f1400c3e640 1 -- 172.21.15.92:0/3529532586 wait complete. 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f1400c3e640 1 Processor -- start 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f1400c3e640 1 -- start start 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f1400c3e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f1400c3e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10b810 con 0x7f13fc107ec0 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f13fa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f13fa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52228/0 (socket says 172.21.15.92:52228) 2024-09-11T22:05:12.565 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f13fa59c640 1 -- 172.21.15.92:0/2810555507 learned_addr learned my addr 172.21.15.92:0/2810555507 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f13fa59c640 1 -- 172.21.15.92:0/2810555507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc19b8b0 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.799+0000 7f13fa59c640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc19b370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0097f0 tx=0x7f13ec030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.800+0000 7f13f37fe640 1 -- 172.21.15.92:0/2810555507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec042070 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.800+0000 7f13f37fe640 1 -- 172.21.15.92:0/2810555507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13ec030d80 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.800+0000 7f13f37fe640 1 -- 172.21.15.92:0/2810555507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13ec037040 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.800+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc19bb40 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.800+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc19e830 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.801+0000 7f13f37fe640 1 -- 172.21.15.92:0/2810555507 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f13ec045070 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.801+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc108320 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.801+0000 7f13f37fe640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 0x7f13e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.801+0000 7f13f37fe640 1 -- 172.21.15.92:0/2810555507 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f13ec06c0c0 con 0x7f13fc107ec0 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.801+0000 7f13f3fff640 1 -- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 msgr2=0x7f13e803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/733069068 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.801+0000 7f13f3fff640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 0x7f13e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:05:12.566 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:11.805+0000 7f13f37fe640 1 -- 172.21.15.92:0/2810555507 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13ec045520 con 0x7f13fc107ec0 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.002+0000 7f13f3fff640 1 -- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 msgr2=0x7f13e803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/733069068 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.002+0000 7f13f3fff640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 0x7f13e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.024+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f13fc19ea00 con 0x7f13fc107ec0 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.025+0000 7f13f37fe640 1 -- 172.21.15.92:0/2810555507 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f13fc19ea00 con 0x7f13fc107ec0 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 msgr2=0x7f13e803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 0x7f13e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 msgr2=0x7f13fc19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc19b370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0097f0 tx=0x7f13ec030c70 comp rx=0 tx=0).stop 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 shutdown_connections 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7f13e8038fc0 0x7f13e803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 --2- 172.21.15.92:0/2810555507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13fc107ec0 0x7f13fc19b370 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 >> 172.21.15.92:0/2810555507 conn(0x7f13fc101ac0 msgr2=0x7f13fc1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:12.567 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 shutdown_connections 2024-09-11T22:05:12.568 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.027+0000 7f1400c3e640 1 -- 172.21.15.92:0/2810555507 wait complete. 2024-09-11T22:05:12.568 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-09-11T22:05:12.568 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 9... 2024-09-11T22:05:12.953 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:12 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2810555507' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T22:05:13.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:12.952+0000 7f8ec9ae0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:05:13.268 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:13.020+0000 7f8ec9ae0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:05:13.768 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:13.729+0000 7f8ec9ae0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:05:14.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:13.846+0000 7f8ec9ae0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:05:14.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:13.989+0000 7f8ec9ae0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:05:14.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.050+0000 7f8ec9ae0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:05:14.202 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.200+0000 7f8ec9ae0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:05:14.486 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.267+0000 7f8ec9ae0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:05:14.486 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.328+0000 7f8ec9ae0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:05:14.486 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.485+0000 7f8ec9ae0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:05:14.759 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.550+0000 7f8ec9ae0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:05:14.759 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.616+0000 7f8ec9ae0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:05:14.759 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.698+0000 7f8ec9ae0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:05:14.759 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.757+0000 7f8ec9ae0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:05:15.018 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:14.816+0000 7f8ec9ae0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:05:16.018 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:15.600+0000 7f8ec9ae0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:05:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: Active manager daemon x restarted 2024-09-11T22:05:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: Activating manager daemon x 2024-09-11T22:05:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T22:05:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: mgrmap e10: x(active, starting, since 0.00338013s) 2024-09-11T22:05:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:05:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:05:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:05:16.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:05:16.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:05:16.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: Manager daemon x is now available 2024-09-11T22:05:16.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:16.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:05:16.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:05:16.769 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:16.635+0000 7f8e34df4640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:05:16] ENGINE Error in HTTPServer.serve 2024-09-11T22:05:16.769 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: Traceback (most recent call last): 2024-09-11T22:05:16.769 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:05:16.769 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self._connections.run(self.expiration_interval) 2024-09-11T22:05:16.769 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:05:16.769 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self._run(expiration_interval) 2024-09-11T22:05:16.769 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: s = self.context.wrap_socket( 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: return self.sslsocket_class._create( 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self.do_handshake() 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: self._sslobj.do_handshake() 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:05:16.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:05:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:05:17.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:05:17.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T22:05:17.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc528d4a640 1 Processor -- start 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc528d4a640 1 -- start start 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc528d4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc52410a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc528d4a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc52407be10 con 0x7fc524107eb0 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc52259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc52410a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc52259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc52410a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52244/0 (socket says 172.21.15.92:52244) 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc52259c640 1 -- 172.21.15.92:0/8626661 learned_addr learned my addr 172.21.15.92:0/8626661 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.877+0000 7fc52259c640 1 -- 172.21.15.92:0/8626661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52410a7e0 con 0x7fc524107eb0 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.878+0000 7fc52259c640 1 --2- 172.21.15.92:0/8626661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc52410a2a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc514009820 tx=0x7fc51402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5f4661be91f1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.878+0000 7fc521d9b640 1 -- 172.21.15.92:0/8626661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc514037070 con 0x7fc524107eb0 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.878+0000 7fc521d9b640 1 -- 172.21.15.92:0/8626661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5140328f0 con 0x7fc524107eb0 2024-09-11T22:05:17.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.878+0000 7fc521d9b640 1 -- 172.21.15.92:0/8626661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc514030740 con 0x7fc524107eb0 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.879+0000 7fc528d4a640 1 -- 172.21.15.92:0/8626661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 msgr2=0x7fc52410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.879+0000 7fc528d4a640 1 --2- 172.21.15.92:0/8626661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc52410a2a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc514009820 tx=0x7fc51402a270 comp rx=0 tx=0).stop 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.879+0000 7fc528d4a640 1 -- 172.21.15.92:0/8626661 shutdown_connections 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.879+0000 7fc528d4a640 1 --2- 172.21.15.92:0/8626661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc52410a2a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.879+0000 7fc528d4a640 1 -- 172.21.15.92:0/8626661 >> 172.21.15.92:0/8626661 conn(0x7fc524101ad0 msgr2=0x7fc524103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.879+0000 7fc528d4a640 1 -- 172.21.15.92:0/8626661 shutdown_connections 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.879+0000 7fc528d4a640 1 -- 172.21.15.92:0/8626661 wait complete. 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc528d4a640 1 Processor -- start 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc528d4a640 1 -- start start 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc528d4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc524107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc528d4a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc52410b800 con 0x7fc524107eb0 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc52259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc524107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:17.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc52259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc524107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52252/0 (socket says 172.21.15.92:52252) 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc52259c640 1 -- 172.21.15.92:0/352025033 learned_addr learned my addr 172.21.15.92:0/352025033 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc52259c640 1 -- 172.21.15.92:0/352025033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc524107c70 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.880+0000 7fc52259c640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc524107730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc514030cc0 tx=0x7fc514030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.881+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc514042070 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.881+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc524105e60 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.881+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc514030e60 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.881+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc514037040 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.881+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5241063a0 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.881+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc514045070 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.882+0000 7fc51b7fe640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 0x7fc51003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.882+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc51406c0c0 con 0x7fc524107eb0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.882+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 --> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc524108310 con 0x7fc510038fc0 2024-09-11T22:05:17.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.882+0000 7fc51bfff640 1 -- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 msgr2=0x7fc51003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/733069068 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:12.882+0000 7fc51bfff640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 0x7fc51003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:13.083+0000 7fc51bfff640 1 -- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 msgr2=0x7fc51003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/733069068 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:13.083+0000 7fc51bfff640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 0x7fc51003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:13.484+0000 7fc51bfff640 1 -- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 msgr2=0x7fc51003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/733069068 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:13.484+0000 7fc51bfff640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 0x7fc51003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:14.285+0000 7fc51bfff640 1 -- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 msgr2=0x7fc51003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/733069068 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:14.285+0000 7fc51bfff640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 0x7fc51003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:15.604+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fc51406ab70 con 0x7fc524107eb0 2024-09-11T22:05:17.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:15.604+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 msgr2=0x7fc51003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:15.604+0000 7fc51b7fe640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 0x7fc51003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.606+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fc51402ad40 con 0x7fc524107eb0 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.606+0000 7fc51b7fe640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc51003c890 0x7fc51003ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.606+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc524108310 con 0x7fc51003c890 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.606+0000 7fc51bfff640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc51003c890 0x7fc51003ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.607+0000 7fc51bfff640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc51003c890 0x7fc51003ec80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc50c003980 tx=0x7fc50c006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.608+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc524108310 con 0x7fc51003c890 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc524102d20 con 0x7fc51003c890 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc51b7fe640 1 -- 172.21.15.92:0/352025033 <== mgr.14150 v2:172.21.15.92:6800/1619835932 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc524102d20 con 0x7fc51003c890 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc51003c890 msgr2=0x7fc51003ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc51003c890 0x7fc51003ec80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc50c003980 tx=0x7fc50c006e10 comp rx=0 tx=0).stop 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 msgr2=0x7fc524107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc524107730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc514030cc0 tx=0x7fc514030cf0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 shutdown_connections 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc51003c890 0x7fc51003ec80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:6800/733069068,v1:172.21.15.92:6801/733069068] conn(0x7fc510038fc0 0x7fc51003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 --2- 172.21.15.92:0/352025033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc524107eb0 0x7fc524107730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 >> 172.21.15.92:0/352025033 conn(0x7fc524101ad0 msgr2=0x7fc524102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 shutdown_connections 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:16.615+0000 7fc528d4a640 1 -- 172.21.15.92:0/352025033 wait complete. 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 9 is available 2024-09-11T22:05:17.288 INFO:teuthology.orchestra.run.smithi092.stdout:Generating a dashboard self-signed certificate... 2024-09-11T22:05:17.871 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: [11/Sep/2024:22:05:16] ENGINE Bus STARTING 2024-09-11T22:05:17.871 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T22:05:17.871 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: mgrmap e11: x(active, since 1.00558s) 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: [11/Sep/2024:22:05:16] ENGINE Serving on https://172.21.15.92:7150 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: [11/Sep/2024:22:05:16] ENGINE Error in HTTPServer.serve 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: Traceback (most recent call last): 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._run(expiration_interval) 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:05:17.872 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: s = self.context.wrap_socket( 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: return self.sslsocket_class._create( 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: self.do_handshake() 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._sslobj.do_handshake() 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: [11/Sep/2024:22:05:16] ENGINE Serving on http://172.21.15.92:8765 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: [11/Sep/2024:22:05:16] ENGINE Bus STARTED 2024-09-11T22:05:17.873 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:17 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:18.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:18.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:18 smithi092 ceph-mon[31173]: mgrmap e12: x(active, since 2s) 2024-09-11T22:05:18.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:18.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T22:05:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55ca087640 1 Processor -- start 2024-09-11T22:05:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55ca087640 1 -- start start 2024-09-11T22:05:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55ca087640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c410a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55ca087640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55c407be10 con 0x7f55c4107ec0 2024-09-11T22:05:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c410a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c410a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54642/0 (socket says 172.21.15.92:54642) 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55c37fe640 1 -- 172.21.15.92:0/208302181 learned_addr learned my addr 172.21.15.92:0/208302181 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.682+0000 7f55c37fe640 1 -- 172.21.15.92:0/208302181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c410a7f0 con 0x7f55c4107ec0 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.683+0000 7f55c37fe640 1 --2- 172.21.15.92:0/208302181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c410a2b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009820 tx=0x7f55b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ccad1b35c1e212 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.683+0000 7f55c2ffd640 1 -- 172.21.15.92:0/208302181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b4037070 con 0x7f55c4107ec0 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.683+0000 7f55c2ffd640 1 -- 172.21.15.92:0/208302181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b40328f0 con 0x7f55c4107ec0 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.683+0000 7f55c2ffd640 1 -- 172.21.15.92:0/208302181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b4030740 con 0x7f55c4107ec0 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 -- 172.21.15.92:0/208302181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 msgr2=0x7f55c410a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 --2- 172.21.15.92:0/208302181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c410a2b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009820 tx=0x7f55b402a270 comp rx=0 tx=0).stop 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 -- 172.21.15.92:0/208302181 shutdown_connections 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 --2- 172.21.15.92:0/208302181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c410a2b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 -- 172.21.15.92:0/208302181 >> 172.21.15.92:0/208302181 conn(0x7f55c4101ac0 msgr2=0x7f55c4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 -- 172.21.15.92:0/208302181 shutdown_connections 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 -- 172.21.15.92:0/208302181 wait complete. 2024-09-11T22:05:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 Processor -- start 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.684+0000 7f55ca087640 1 -- start start 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55ca087640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c4107740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55ca087640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55c410b810 con 0x7f55c4107ec0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c4107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c4107740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54654/0 (socket says 172.21.15.92:54654) 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c37fe640 1 -- 172.21.15.92:0/3419493104 learned_addr learned my addr 172.21.15.92:0/3419493104 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c37fe640 1 -- 172.21.15.92:0/3419493104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c4107c80 con 0x7f55c4107ec0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c37fe640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c4107740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55b4030cc0 tx=0x7f55b4030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b4042070 con 0x7f55c4107ec0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b4030e60 con 0x7f55c4107ec0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c4105e70 con 0x7f55c4107ec0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b4037040 con 0x7f55c4107ec0 2024-09-11T22:05:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.685+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41063b0 con 0x7f55c4107ec0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.686+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f55b4045070 con 0x7f55c4107ec0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.686+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c4108320 con 0x7f55c4107ec0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.686+0000 7f55c17fa640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f55b0038f00 0x7f55b003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.686+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f55b406c0b0 con 0x7f55c4107ec0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.686+0000 7f55bbfff640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f55b0038f00 0x7f55b003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.687+0000 7f55bbfff640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f55b0038f00 0x7f55b003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098c0 tx=0x7f55ac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.691+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55b403be10 con 0x7f55c4107ec0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.834+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f55b403cb70 con 0x7f55c4107ec0 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:17.871+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f55c4106d00 con 0x7f55b0038f00 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.130+0000 7f55c17fa640 1 -- 172.21.15.92:0/3419493104 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f55c4106d00 con 0x7f55b0038f00 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f55b0038f00 msgr2=0x7f55b003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f55b0038f00 0x7f55b003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098c0 tx=0x7f55ac006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.136 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 msgr2=0x7f55c4107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c4107740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55b4030cc0 tx=0x7f55b4030cf0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 shutdown_connections 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f55b0038f00 0x7f55b003b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 --2- 172.21.15.92:0/3419493104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c4107ec0 0x7f55c4107740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.132+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 >> 172.21.15.92:0/3419493104 conn(0x7f55c4101ac0 msgr2=0x7f55c41025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.133+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 shutdown_connections 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:18.133+0000 7f55ca087640 1 -- 172.21.15.92:0/3419493104 wait complete. 2024-09-11T22:05:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial admin user... 2024-09-11T22:05:19.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:19 smithi092 ceph-mon[31173]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:20.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$atveUDcJM.RYUXkuvxvM7eyOE97e0DzjzFfo0bTp4bRnu8NouViGu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726092319, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T22:05:20.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.525+0000 7faeb9feb640 1 Processor -- start 2024-09-11T22:05:20.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.525+0000 7faeb9feb640 1 -- start start 2024-09-11T22:05:20.491 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.526+0000 7faeb9feb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb410a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.526+0000 7faeb9feb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4075880 con 0x7faeb4107e90 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.526+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb410a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.526+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb410a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54664/0 (socket says 172.21.15.92:54664) 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.526+0000 7faeb37fe640 1 -- 172.21.15.92:0/3019973408 learned_addr learned my addr 172.21.15.92:0/3019973408 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.526+0000 7faeb37fe640 1 -- 172.21.15.92:0/3019973408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb410a7c0 con 0x7faeb4107e90 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.527+0000 7faeb37fe640 1 --2- 172.21.15.92:0/3019973408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb410a280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faea4009820 tx=0x7faea402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c979eafda0a5b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.527+0000 7faeb2ffd640 1 -- 172.21.15.92:0/3019973408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea4037070 con 0x7faeb4107e90 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.527+0000 7faeb2ffd640 1 -- 172.21.15.92:0/3019973408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea40328f0 con 0x7faeb4107e90 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.527+0000 7faeb2ffd640 1 -- 172.21.15.92:0/3019973408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea4030740 con 0x7faeb4107e90 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 -- 172.21.15.92:0/3019973408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 msgr2=0x7faeb410a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 --2- 172.21.15.92:0/3019973408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb410a280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faea4009820 tx=0x7faea402a270 comp rx=0 tx=0).stop 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 -- 172.21.15.92:0/3019973408 shutdown_connections 2024-09-11T22:05:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 --2- 172.21.15.92:0/3019973408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb410a280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 -- 172.21.15.92:0/3019973408 >> 172.21.15.92:0/3019973408 conn(0x7faeb4101a90 msgr2=0x7faeb4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 -- 172.21.15.92:0/3019973408 shutdown_connections 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 -- 172.21.15.92:0/3019973408 wait complete. 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 Processor -- start 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.528+0000 7faeb9feb640 1 -- start start 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb9feb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb4107710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb9feb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410b840 con 0x7faeb4107e90 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb4107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb4107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54672/0 (socket says 172.21.15.92:54672) 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb37fe640 1 -- 172.21.15.92:0/3137606534 learned_addr learned my addr 172.21.15.92:0/3137606534 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb37fe640 1 -- 172.21.15.92:0/3137606534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb4107c50 con 0x7faeb4107e90 2024-09-11T22:05:20.493 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb37fe640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb4107710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faea4030cc0 tx=0x7faea4030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb17fa640 1 -- 172.21.15.92:0/3137606534 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea4042070 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb17fa640 1 -- 172.21.15.92:0/3137606534 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea4030e60 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb17fa640 1 -- 172.21.15.92:0/3137606534 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea4037040 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb4105e40 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.529+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb4106380 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.530+0000 7faeb17fa640 1 -- 172.21.15.92:0/3137606534 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7faea4045070 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.530+0000 7faeb17fa640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faea0038fc0 0x7faea003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.530+0000 7faeb17fa640 1 -- 172.21.15.92:0/3137606534 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faea406c170 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.530+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb41082f0 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.531+0000 7faeaaffd640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faea0038fc0 0x7faea003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.531+0000 7faeaaffd640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faea0038fc0 0x7faea003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098c0 tx=0x7fae9c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.535+0000 7faeb17fa640 1 -- 172.21.15.92:0/3137606534 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faea402ed90 con 0x7faeb4107e90 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.717+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7faeb4106c80 con 0x7faea0038fc0 2024-09-11T22:05:20.494 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.941+0000 7faeb17fa640 1 -- 172.21.15.92:0/3137606534 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7faeb4106c80 con 0x7faea0038fc0 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faea0038fc0 msgr2=0x7faea003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faea0038fc0 0x7faea003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098c0 tx=0x7fae9c006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 msgr2=0x7faeb4107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb4107710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faea4030cc0 tx=0x7faea4030cf0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 shutdown_connections 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faea0038fc0 0x7faea003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 --2- 172.21.15.92:0/3137606534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeb4107e90 0x7faeb4107710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 >> 172.21.15.92:0/3137606534 conn(0x7faeb4101a90 msgr2=0x7faeb41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 shutdown_connections 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:19.943+0000 7faeb9feb640 1 -- 172.21.15.92:0/3137606534 wait complete. 2024-09-11T22:05:20.495 INFO:teuthology.orchestra.run.smithi092.stdout:Fetching dashboard port number... 2024-09-11T22:05:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:20 smithi092 ceph-mon[31173]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:21.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.899+0000 7fc32d9ed640 1 Processor -- start 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.899+0000 7fc32d9ed640 1 -- start start 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.899+0000 7fc32d9ed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32810a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.899+0000 7fc32d9ed640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc32807be10 con 0x7fc328107ec0 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.900+0000 7fc326ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32810a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.900+0000 7fc326ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32810a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54684/0 (socket says 172.21.15.92:54684) 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.900+0000 7fc326ffd640 1 -- 172.21.15.92:0/1671609504 learned_addr learned my addr 172.21.15.92:0/1671609504 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.900+0000 7fc326ffd640 1 -- 172.21.15.92:0/1671609504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32810a7f0 con 0x7fc328107ec0 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.901+0000 7fc326ffd640 1 --2- 172.21.15.92:0/1671609504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32810a2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc318009820 tx=0x7fc31802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e1f077f9604a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.901+0000 7fc325ffb640 1 -- 172.21.15.92:0/1671609504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc318037070 con 0x7fc328107ec0 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.901+0000 7fc325ffb640 1 -- 172.21.15.92:0/1671609504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3180328f0 con 0x7fc328107ec0 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.901+0000 7fc325ffb640 1 -- 172.21.15.92:0/1671609504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc318030740 con 0x7fc328107ec0 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 -- 172.21.15.92:0/1671609504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 msgr2=0x7fc32810a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 --2- 172.21.15.92:0/1671609504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32810a2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc318009820 tx=0x7fc31802a270 comp rx=0 tx=0).stop 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 -- 172.21.15.92:0/1671609504 shutdown_connections 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 --2- 172.21.15.92:0/1671609504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32810a2b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 -- 172.21.15.92:0/1671609504 >> 172.21.15.92:0/1671609504 conn(0x7fc328101ae0 msgr2=0x7fc328103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 -- 172.21.15.92:0/1671609504 shutdown_connections 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 -- 172.21.15.92:0/1671609504 wait complete. 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 Processor -- start 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.902+0000 7fc32d9ed640 1 -- start start 2024-09-11T22:05:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc32d9ed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc32d9ed640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc32810b810 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc326ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc326ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54690/0 (socket says 172.21.15.92:54690) 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc326ffd640 1 -- 172.21.15.92:0/604308367 learned_addr learned my addr 172.21.15.92:0/604308367 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc326ffd640 1 -- 172.21.15.92:0/604308367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32819b8a0 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc326ffd640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32819b360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3180097f0 tx=0x7fc318030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc32c9eb640 1 -- 172.21.15.92:0/604308367 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc318042070 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc32c9eb640 1 -- 172.21.15.92:0/604308367 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc318030d80 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32819bb30 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc32c9eb640 1 -- 172.21.15.92:0/604308367 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc318037040 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.903+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32819e820 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.904+0000 7fc32c9eb640 1 -- 172.21.15.92:0/604308367 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fc318045070 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.904+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc328108320 con 0x7fc328107ec0 2024-09-11T22:05:21.633 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.905+0000 7fc32c9eb640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc314038fc0 0x7fc31403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.905+0000 7fc32c9eb640 1 -- 172.21.15.92:0/604308367 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc31806c1a0 con 0x7fc328107ec0 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.905+0000 7fc3267fc640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc314038fc0 0x7fc31403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.905+0000 7fc3267fc640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc314038fc0 0x7fc31403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098c0 tx=0x7fc310006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:20.909+0000 7fc32c9eb640 1 -- 172.21.15.92:0/604308367 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc318045520 con 0x7fc328107ec0 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.080+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc32819ee10 con 0x7fc328107ec0 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.080+0000 7fc32c9eb640 1 -- 172.21.15.92:0/604308367 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc31803add0 con 0x7fc328107ec0 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc314038fc0 msgr2=0x7fc31403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc314038fc0 0x7fc31403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098c0 tx=0x7fc310006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 msgr2=0x7fc32819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32819b360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3180097f0 tx=0x7fc318030c70 comp rx=0 tx=0).stop 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 shutdown_connections 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc314038fc0 0x7fc31403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 --2- 172.21.15.92:0/604308367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc328107ec0 0x7fc32819b360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:21.634 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.082+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 >> 172.21.15.92:0/604308367 conn(0x7fc328101ae0 msgr2=0x7fc328102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:21.635 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.083+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 shutdown_connections 2024-09-11T22:05:21.635 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:21.083+0000 7fc32d9ed640 1 -- 172.21.15.92:0/604308367 wait complete. 2024-09-11T22:05:21.638 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:05:21.638 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T22:05:21.646 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:05:21.646 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T22:05:21.646 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T22:05:21.646 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T22:05:21.647 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph Dashboard is now available at: 2024-09-11T22:05:21.647 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:21.647 INFO:teuthology.orchestra.run.smithi092.stdout: URL: https://smithi092.front.sepia.ceph.com:8443/ 2024-09-11T22:05:21.647 INFO:teuthology.orchestra.run.smithi092.stdout: User: admin 2024-09-11T22:05:21.647 INFO:teuthology.orchestra.run.smithi092.stdout: Password: 93t9q6hb47 2024-09-11T22:05:21.648 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:21.648 INFO:teuthology.orchestra.run.smithi092.stdout:Saving cluster configuration to /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config directory 2024-09-11T22:05:22.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:21 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/604308367' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T22:05:22.847 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:22 smithi092 ceph-mon[31173]: mgrmap e13: x(active, since 6s) 2024-09-11T22:05:22.847 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:22 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4102621944' entity='client.admin' 2024-09-11T22:05:22.855 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdb276f640 1 Processor -- start 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdb276f640 1 -- start start 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdb276f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdb276f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efdac106ce0 con 0x7efdac106310 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdabfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdabfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54704/0 (socket says 172.21.15.92:54704) 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdabfff640 1 -- 172.21.15.92:0/3093034507 learned_addr learned my addr 172.21.15.92:0/3093034507 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.039+0000 7efdabfff640 1 -- 172.21.15.92:0/3093034507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac107510 con 0x7efdac106310 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.040+0000 7efdabfff640 1 --2- 172.21.15.92:0/3093034507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac106710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00bac0 tx=0x7efd9c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6799904787757c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.040+0000 7efdaaffd640 1 -- 172.21.15.92:0/3093034507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd9c0326d0 con 0x7efdac106310 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.040+0000 7efdaaffd640 1 -- 172.21.15.92:0/3093034507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd9c038070 con 0x7efdac106310 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.040+0000 7efdaaffd640 1 -- 172.21.15.92:0/3093034507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd9c033c40 con 0x7efdac106310 2024-09-11T22:05:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 -- 172.21.15.92:0/3093034507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 msgr2=0x7efdac106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 --2- 172.21.15.92:0/3093034507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac106710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00bac0 tx=0x7efd9c02c920 comp rx=0 tx=0).stop 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 -- 172.21.15.92:0/3093034507 shutdown_connections 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 --2- 172.21.15.92:0/3093034507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac106710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 -- 172.21.15.92:0/3093034507 >> 172.21.15.92:0/3093034507 conn(0x7efdac101ae0 msgr2=0x7efdac103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 -- 172.21.15.92:0/3093034507 shutdown_connections 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 -- 172.21.15.92:0/3093034507 wait complete. 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 Processor -- start 2024-09-11T22:05:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.041+0000 7efdb276f640 1 -- start start 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdb276f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdb276f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efdac07b0f0 con 0x7efdac106310 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdabfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdabfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54720/0 (socket says 172.21.15.92:54720) 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdabfff640 1 -- 172.21.15.92:0/4102621944 learned_addr learned my addr 172.21.15.92:0/4102621944 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdabfff640 1 -- 172.21.15.92:0/4102621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac199710 con 0x7efdac106310 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdabfff640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac1991d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00b2c0 tx=0x7efd9c0327d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efda97fa640 1 -- 172.21.15.92:0/4102621944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd9c042070 con 0x7efdac106310 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac1999a0 con 0x7efdac106310 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efda97fa640 1 -- 172.21.15.92:0/4102621944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd9c032d20 con 0x7efdac106310 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.042+0000 7efda97fa640 1 -- 172.21.15.92:0/4102621944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd9c03c580 con 0x7efdac106310 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.043+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac19c690 con 0x7efdac106310 2024-09-11T22:05:22.858 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.043+0000 7efda97fa640 1 -- 172.21.15.92:0/4102621944 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efd9c03c880 con 0x7efdac106310 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.044+0000 7efda97fa640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efd94039010 0x7efd9403b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.044+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdac106710 con 0x7efdac106310 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.044+0000 7efda97fa640 1 -- 172.21.15.92:0/4102621944 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd9c030050 con 0x7efdac106310 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.044+0000 7efdab7fe640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efd94039010 0x7efd9403b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.044+0000 7efdab7fe640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efd94039010 0x7efd9403b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efda00098c0 tx=0x7efda0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.048+0000 7efda97fa640 1 -- 172.21.15.92:0/4102621944 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd9c054080 con 0x7efdac106310 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.300+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7efdac19c860 con 0x7efdac106310 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.302+0000 7efda97fa640 1 -- 172.21.15.92:0/4102621944 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7efdac19c860 con 0x7efdac106310 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efd94039010 msgr2=0x7efd9403b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efd94039010 0x7efd9403b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efda00098c0 tx=0x7efda0006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 msgr2=0x7efdac1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac1991d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00b2c0 tx=0x7efd9c0327d0 comp rx=0 tx=0).stop 2024-09-11T22:05:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 shutdown_connections 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efd94039010 0x7efd9403b4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 --2- 172.21.15.92:0/4102621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdac106310 0x7efdac1991d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 >> 172.21.15.92:0/4102621944 conn(0x7efdac101ae0 msgr2=0x7efdac18f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 shutdown_connections 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-09-11T22:05:22.304+0000 7efdb276f640 1 -- 172.21.15.92:0/4102621944 wait complete. 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.860 INFO:teuthology.orchestra.run.smithi092.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout: ceph telemetry on 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout:For more information see: 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:22.861 INFO:teuthology.orchestra.run.smithi092.stdout:Bootstrap complete. 2024-09-11T22:05:22.896 INFO:tasks.cephadm:Fetching config... 2024-09-11T22:05:22.897 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:05:22.897 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T22:05:22.914 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T22:05:22.914 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:05:22.914 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T22:05:22.970 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T22:05:22.970 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:05:22.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T22:05:23.036 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T22:05:23.036 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:05:23.036 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T22:05:23.091 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T22:05:23.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPZOjNyZK2EuKPwdf02uZ7dCcZZsxYK2dHGmlkAdEfA96S6ToHQf/4wwzjjAWKHZyKwTjVLe5mR0eY3OaXKmtrLmNq8idjM+FHYGJe6l692QhKKxjp1fo3DS/kLdw+cT95PVlgC6Rd43C2EwWWta4vKW+znQlXRCVsrmMQXrL09obYX763gJHP7zFCSYWnYjpXiGeyn032zFHeo63K84bpONl1z6s71uC3RUOShldhnPJWCJWHQBXl8xv8DtfnhFLhCPn37p3U2DZ5JNJ0vRxeJ6uHadJ/xoxm2U/euS0dmmt4myzV20GBB7KvdVeYK10iTKnFtgQKEKorvrPmSlVwjrFH/vNCAg8cmaFv6GzyldP9QB1yw9+2KtgWguwseTaNKVeHY2+Vea41J3sn4k2rKy06glPfFVCAV+GLTTmj3nU3v8boZtAfj1ZqPAAWjhbx24qfwZyb4IVKfYKe0aT+ADVXw9D8ckdewUFrV4PFIP6RJAvEbASzlV3VtTmarcs= ceph-a6709e28-7089-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:05:23.165 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPZOjNyZK2EuKPwdf02uZ7dCcZZsxYK2dHGmlkAdEfA96S6ToHQf/4wwzjjAWKHZyKwTjVLe5mR0eY3OaXKmtrLmNq8idjM+FHYGJe6l692QhKKxjp1fo3DS/kLdw+cT95PVlgC6Rd43C2EwWWta4vKW+znQlXRCVsrmMQXrL09obYX763gJHP7zFCSYWnYjpXiGeyn032zFHeo63K84bpONl1z6s71uC3RUOShldhnPJWCJWHQBXl8xv8DtfnhFLhCPn37p3U2DZ5JNJ0vRxeJ6uHadJ/xoxm2U/euS0dmmt4myzV20GBB7KvdVeYK10iTKnFtgQKEKorvrPmSlVwjrFH/vNCAg8cmaFv6GzyldP9QB1yw9+2KtgWguwseTaNKVeHY2+Vea41J3sn4k2rKy06glPfFVCAV+GLTTmj3nU3v8boZtAfj1ZqPAAWjhbx24qfwZyb4IVKfYKe0aT+ADVXw9D8ckdewUFrV4PFIP6RJAvEbASzlV3VtTmarcs= ceph-a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:05:23.176 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPZOjNyZK2EuKPwdf02uZ7dCcZZsxYK2dHGmlkAdEfA96S6ToHQf/4wwzjjAWKHZyKwTjVLe5mR0eY3OaXKmtrLmNq8idjM+FHYGJe6l692QhKKxjp1fo3DS/kLdw+cT95PVlgC6Rd43C2EwWWta4vKW+znQlXRCVsrmMQXrL09obYX763gJHP7zFCSYWnYjpXiGeyn032zFHeo63K84bpONl1z6s71uC3RUOShldhnPJWCJWHQBXl8xv8DtfnhFLhCPn37p3U2DZ5JNJ0vRxeJ6uHadJ/xoxm2U/euS0dmmt4myzV20GBB7KvdVeYK10iTKnFtgQKEKorvrPmSlVwjrFH/vNCAg8cmaFv6GzyldP9QB1yw9+2KtgWguwseTaNKVeHY2+Vea41J3sn4k2rKy06glPfFVCAV+GLTTmj3nU3v8boZtAfj1ZqPAAWjhbx24qfwZyb4IVKfYKe0aT+ADVXw9D8ckdewUFrV4PFIP6RJAvEbASzlV3VtTmarcs= ceph-a6709e28-7089-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:05:23.314 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPZOjNyZK2EuKPwdf02uZ7dCcZZsxYK2dHGmlkAdEfA96S6ToHQf/4wwzjjAWKHZyKwTjVLe5mR0eY3OaXKmtrLmNq8idjM+FHYGJe6l692QhKKxjp1fo3DS/kLdw+cT95PVlgC6Rd43C2EwWWta4vKW+znQlXRCVsrmMQXrL09obYX763gJHP7zFCSYWnYjpXiGeyn032zFHeo63K84bpONl1z6s71uC3RUOShldhnPJWCJWHQBXl8xv8DtfnhFLhCPn37p3U2DZ5JNJ0vRxeJ6uHadJ/xoxm2U/euS0dmmt4myzV20GBB7KvdVeYK10iTKnFtgQKEKorvrPmSlVwjrFH/vNCAg8cmaFv6GzyldP9QB1yw9+2KtgWguwseTaNKVeHY2+Vea41J3sn4k2rKy06glPfFVCAV+GLTTmj3nU3v8boZtAfj1ZqPAAWjhbx24qfwZyb4IVKfYKe0aT+ADVXw9D8ckdewUFrV4PFIP6RJAvEbASzlV3VtTmarcs= ceph-a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:05:23.328 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPZOjNyZK2EuKPwdf02uZ7dCcZZsxYK2dHGmlkAdEfA96S6ToHQf/4wwzjjAWKHZyKwTjVLe5mR0eY3OaXKmtrLmNq8idjM+FHYGJe6l692QhKKxjp1fo3DS/kLdw+cT95PVlgC6Rd43C2EwWWta4vKW+znQlXRCVsrmMQXrL09obYX763gJHP7zFCSYWnYjpXiGeyn032zFHeo63K84bpONl1z6s71uC3RUOShldhnPJWCJWHQBXl8xv8DtfnhFLhCPn37p3U2DZ5JNJ0vRxeJ6uHadJ/xoxm2U/euS0dmmt4myzV20GBB7KvdVeYK10iTKnFtgQKEKorvrPmSlVwjrFH/vNCAg8cmaFv6GzyldP9QB1yw9+2KtgWguwseTaNKVeHY2+Vea41J3sn4k2rKy06glPfFVCAV+GLTTmj3nU3v8boZtAfj1ZqPAAWjhbx24qfwZyb4IVKfYKe0aT+ADVXw9D8ckdewUFrV4PFIP6RJAvEbASzlV3VtTmarcs= ceph-a6709e28-7089-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:05:23.471 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPZOjNyZK2EuKPwdf02uZ7dCcZZsxYK2dHGmlkAdEfA96S6ToHQf/4wwzjjAWKHZyKwTjVLe5mR0eY3OaXKmtrLmNq8idjM+FHYGJe6l692QhKKxjp1fo3DS/kLdw+cT95PVlgC6Rd43C2EwWWta4vKW+znQlXRCVsrmMQXrL09obYX763gJHP7zFCSYWnYjpXiGeyn032zFHeo63K84bpONl1z6s71uC3RUOShldhnPJWCJWHQBXl8xv8DtfnhFLhCPn37p3U2DZ5JNJ0vRxeJ6uHadJ/xoxm2U/euS0dmmt4myzV20GBB7KvdVeYK10iTKnFtgQKEKorvrPmSlVwjrFH/vNCAg8cmaFv6GzyldP9QB1yw9+2KtgWguwseTaNKVeHY2+Vea41J3sn4k2rKy06glPfFVCAV+GLTTmj3nU3v8boZtAfj1ZqPAAWjhbx24qfwZyb4IVKfYKe0aT+ADVXw9D8ckdewUFrV4PFIP6RJAvEbASzlV3VtTmarcs= ceph-a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:05:23.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T22:05:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:24.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.348+0000 7f5ef7e29640 1 -- 172.21.15.92:0/3524168196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 msgr2=0x7f5ef0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.348+0000 7f5ef7e29640 1 --2- 172.21.15.92:0/3524168196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0101a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009880 tx=0x7f5eec02a5d0 comp rx=0 tx=0).stop 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.348+0000 7f5ef7e29640 1 -- 172.21.15.92:0/3524168196 shutdown_connections 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.348+0000 7f5ef7e29640 1 --2- 172.21.15.92:0/3524168196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0101a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.348+0000 7f5ef7e29640 1 -- 172.21.15.92:0/3524168196 >> 172.21.15.92:0/3524168196 conn(0x7f5ef00fcdb0 msgr2=0x7f5ef00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.348+0000 7f5ef7e29640 1 -- 172.21.15.92:0/3524168196 shutdown_connections 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.348+0000 7f5ef7e29640 1 -- 172.21.15.92:0/3524168196 wait complete. 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef7e29640 1 Processor -- start 2024-09-11T22:05:24.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef7e29640 1 -- start start 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef7e29640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef7e29640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01075c0 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef5bc5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef5bc5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54734/0 (socket says 172.21.15.92:54734) 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef5bc5640 1 -- 172.21.15.92:0/1400193241 learned_addr learned my addr 172.21.15.92:0/1400193241 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.349+0000 7f5ef5bc5640 1 -- 172.21.15.92:0/1400193241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef00763f0 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.350+0000 7f5ef5bc5640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009230 tx=0x7f5eec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.350+0000 7f5ee6ffd640 1 -- 172.21.15.92:0/1400193241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eec03f070 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.350+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef00745e0 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.350+0000 7f5ee6ffd640 1 -- 172.21.15.92:0/1400193241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eec003aa0 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.350+0000 7f5ee6ffd640 1 -- 172.21.15.92:0/1400193241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eec035030 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.350+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef0074b20 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.351+0000 7f5ee6ffd640 1 -- 172.21.15.92:0/1400193241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5eec042070 con 0x7f5ef0101600 2024-09-11T22:05:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.351+0000 7f5ee6ffd640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5edc039060 0x7f5edc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.351+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0107d40 con 0x7f5ef0101600 2024-09-11T22:05:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.351+0000 7f5ee6ffd640 1 -- 172.21.15.92:0/1400193241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5eec06ce30 con 0x7f5ef0101600 2024-09-11T22:05:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.351+0000 7f5ef53c4640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5edc039060 0x7f5edc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.351+0000 7f5ef53c4640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5edc039060 0x7f5edc03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00098c0 tx=0x7f5ee0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.355+0000 7f5ee6ffd640 1 -- 172.21.15.92:0/1400193241 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5eec039350 con 0x7f5ef0101600 2024-09-11T22:05:24.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.525+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5ef0190040 con 0x7f5ef0101600 2024-09-11T22:05:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.527+0000 7f5ee6ffd640 1 -- 172.21.15.92:0/1400193241 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5eec04a260 con 0x7f5ef0101600 2024-09-11T22:05:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5edc039060 msgr2=0x7f5edc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5edc039060 0x7f5edc03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00098c0 tx=0x7f5ee0006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 msgr2=0x7f5ef0075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009230 tx=0x7f5eec0048f0 comp rx=0 tx=0).stop 2024-09-11T22:05:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 shutdown_connections 2024-09-11T22:05:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5edc039060 0x7f5edc03b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 --2- 172.21.15.92:0/1400193241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ef0101600 0x7f5ef0075eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 >> 172.21.15.92:0/1400193241 conn(0x7f5ef00fcdb0 msgr2=0x7f5ef00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 shutdown_connections 2024-09-11T22:05:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:24.530+0000 7f5ef7e29640 1 -- 172.21.15.92:0/1400193241 wait complete. 2024-09-11T22:05:25.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:25 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1400193241' entity='client.admin' 2024-09-11T22:05:25.916 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T22:05:25.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T22:05:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.030+0000 7fb212bb3640 1 -- 172.21.15.92:0/4252221498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 msgr2=0x7fb20c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.030+0000 7fb212bb3640 1 --2- 172.21.15.92:0/4252221498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c1008e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009880 tx=0x7fb1fc02a5d0 comp rx=0 tx=0).stop 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.030+0000 7fb212bb3640 1 -- 172.21.15.92:0/4252221498 shutdown_connections 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.030+0000 7fb212bb3640 1 --2- 172.21.15.92:0/4252221498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c1008e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.030+0000 7fb212bb3640 1 -- 172.21.15.92:0/4252221498 >> 172.21.15.92:0/4252221498 conn(0x7fb20c0fbc90 msgr2=0x7fb20c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.030+0000 7fb212bb3640 1 -- 172.21.15.92:0/4252221498 shutdown_connections 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.030+0000 7fb212bb3640 1 -- 172.21.15.92:0/4252221498 wait complete. 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.031+0000 7fb212bb3640 1 Processor -- start 2024-09-11T22:05:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.031+0000 7fb212bb3640 1 -- start start 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.031+0000 7fb212bb3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c0fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.031+0000 7fb212bb3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1064a0 con 0x7fb20c1004e0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.031+0000 7fb21094f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c0fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.031+0000 7fb21094f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c0fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35066/0 (socket says 172.21.15.92:35066) 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.031+0000 7fb21094f640 1 -- 172.21.15.92:0/2344961841 learned_addr learned my addr 172.21.15.92:0/2344961841 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.032+0000 7fb21094f640 1 -- 172.21.15.92:0/2344961841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c0fe040 con 0x7fb20c1004e0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.032+0000 7fb21094f640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c0fdb00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009230 tx=0x7fb1fc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.032+0000 7fb209ffb640 1 -- 172.21.15.92:0/2344961841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1fc03f070 con 0x7fb20c1004e0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.032+0000 7fb209ffb640 1 -- 172.21.15.92:0/2344961841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1fc003aa0 con 0x7fb20c1004e0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.032+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c0fc230 con 0x7fb20c1004e0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.032+0000 7fb209ffb640 1 -- 172.21.15.92:0/2344961841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1fc035030 con 0x7fb20c1004e0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.032+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c0fc770 con 0x7fb20c1004e0 2024-09-11T22:05:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.033+0000 7fb209ffb640 1 -- 172.21.15.92:0/2344961841 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb1fc042070 con 0x7fb20c1004e0 2024-09-11T22:05:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.033+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c106c20 con 0x7fb20c1004e0 2024-09-11T22:05:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.033+0000 7fb209ffb640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb1f4039060 0x7fb1f403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.033+0000 7fb209ffb640 1 -- 172.21.15.92:0/2344961841 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb1fc06ce50 con 0x7fb20c1004e0 2024-09-11T22:05:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.033+0000 7fb20bfff640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb1f4039060 0x7fb1f403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.034+0000 7fb20bfff640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb1f4039060 0x7fb1f403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2000098c0 tx=0x7fb200006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.038+0000 7fb209ffb640 1 -- 172.21.15.92:0/2344961841 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1fc02f300 con 0x7fb20c1004e0 2024-09-11T22:05:27.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.205+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb20c190230 con 0x7fb1f4039060 2024-09-11T22:05:27.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.208+0000 7fb209ffb640 1 -- 172.21.15.92:0/2344961841 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb20c190230 con 0x7fb1f4039060 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.210+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb1f4039060 msgr2=0x7fb1f403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.210+0000 7fb212bb3640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb1f4039060 0x7fb1f403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2000098c0 tx=0x7fb200006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.210+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 msgr2=0x7fb20c0fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.210+0000 7fb212bb3640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c0fdb00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009230 tx=0x7fb1fc0048f0 comp rx=0 tx=0).stop 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.210+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 shutdown_connections 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.211+0000 7fb212bb3640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb1f4039060 0x7fb1f403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.211+0000 7fb212bb3640 1 --2- 172.21.15.92:0/2344961841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb20c1004e0 0x7fb20c0fdb00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.211+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 >> 172.21.15.92:0/2344961841 conn(0x7fb20c0fbc90 msgr2=0x7fb20c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.211+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 shutdown_connections 2024-09-11T22:05:27.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:27.211+0000 7fb212bb3640 1 -- 172.21.15.92:0/2344961841 wait complete. 2024-09-11T22:05:27.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:27.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:27.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:05:27.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:27.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:27.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:27.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:27.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:27.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:27.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:05:27.936 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi110 2024-09-11T22:05:27.936 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:05:27.936 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.conf 2024-09-11T22:05:27.956 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:05:27.956 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:05:28.014 INFO:tasks.cephadm:Adding host smithi110 to orchestrator... 2024-09-11T22:05:28.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch host add smithi110 2024-09-11T22:05:28.256 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:28.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:28 smithi092 ceph-mon[31173]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:28.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:28 smithi092 ceph-mon[31173]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T22:05:28.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:28 smithi092 ceph-mon[31173]: Updating smithi092:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:05:28.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:28 smithi092 ceph-mon[31173]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:05:28.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:28.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:28.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:28.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.671+0000 7f443e771640 1 -- 172.21.15.92:0/2289175542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 msgr2=0x7f4438107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.671+0000 7f443e771640 1 --2- 172.21.15.92:0/2289175542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f4438107a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4428009880 tx=0x7f442802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.671+0000 7f443e771640 1 -- 172.21.15.92:0/2289175542 shutdown_connections 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.671+0000 7f443e771640 1 --2- 172.21.15.92:0/2289175542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f4438107a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.671+0000 7f443e771640 1 -- 172.21.15.92:0/2289175542 >> 172.21.15.92:0/2289175542 conn(0x7f44380fd2b0 msgr2=0x7f44380ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.671+0000 7f443e771640 1 -- 172.21.15.92:0/2289175542 shutdown_connections 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.672+0000 7f443e771640 1 -- 172.21.15.92:0/2289175542 wait complete. 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.672+0000 7f443e771640 1 Processor -- start 2024-09-11T22:05:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.672+0000 7f443e771640 1 -- start start 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f443e771640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f44380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f443e771640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f443810bb60 con 0x7f4438107680 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f4437fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f44380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f4437fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f44380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35090/0 (socket says 172.21.15.92:35090) 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f4437fff640 1 -- 172.21.15.92:0/1422152316 learned_addr learned my addr 172.21.15.92:0/1422152316 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f4437fff640 1 -- 172.21.15.92:0/1422152316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44380763e0 con 0x7f4438107680 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f4437fff640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f44380744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4428009230 tx=0x7f44280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f44357fa640 1 -- 172.21.15.92:0/1422152316 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f442803f070 con 0x7f4438107680 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f44357fa640 1 -- 172.21.15.92:0/1422152316 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4428004d10 con 0x7f4438107680 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4438074a30 con 0x7f4438107680 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f44357fa640 1 -- 172.21.15.92:0/1422152316 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428035030 con 0x7f4438107680 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.673+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4438074ef0 con 0x7f4438107680 2024-09-11T22:05:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.674+0000 7f44357fa640 1 -- 172.21.15.92:0/1422152316 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4428042070 con 0x7f4438107680 2024-09-11T22:05:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.674+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438102cf0 con 0x7f4438107680 2024-09-11T22:05:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.674+0000 7f44357fa640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f44240390c0 0x7f442403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.674+0000 7f44357fa640 1 -- 172.21.15.92:0/1422152316 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f442806cd70 con 0x7f4438107680 2024-09-11T22:05:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.674+0000 7f44377fe640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f44240390c0 0x7f442403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.675+0000 7f44377fe640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f44240390c0 0x7f442403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f44200098c0 tx=0x7f4420006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.679+0000 7f44357fa640 1 -- 172.21.15.92:0/1422152316 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4428039350 con 0x7f4438107680 2024-09-11T22:05:28.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:28.851+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}) -- 0x7f4438075a90 con 0x7f44240390c0 2024-09-11T22:05:30.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:29 smithi092 ceph-mon[31173]: Updating smithi092:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:05:31.029 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:30 smithi092 ceph-mon[31173]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:31.030 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:30 smithi092 ceph-mon[31173]: Deploying cephadm binary to smithi110 2024-09-11T22:05:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.997+0000 7f44357fa640 1 -- 172.21.15.92:0/1422152316 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4438075a90 con 0x7f44240390c0 2024-09-11T22:05:31.997 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi110' with addr '172.21.15.110' 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f44240390c0 msgr2=0x7f442403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f44240390c0 0x7f442403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f44200098c0 tx=0x7f4420006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 msgr2=0x7f44380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f44380744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4428009230 tx=0x7f44280048f0 comp rx=0 tx=0).stop 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 shutdown_connections 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f44240390c0 0x7f442403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 --2- 172.21.15.92:0/1422152316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438107680 0x7f44380744f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:31.999+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 >> 172.21.15.92:0/1422152316 conn(0x7f44380fd2b0 msgr2=0x7f44380fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:32.000+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 shutdown_connections 2024-09-11T22:05:32.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:32.000+0000 7f443e771640 1 -- 172.21.15.92:0/1422152316 wait complete. 2024-09-11T22:05:32.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T22:05:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:33.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:33.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:32 smithi092 ceph-mon[31173]: Added host smithi110 2024-09-11T22:05:33.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:33.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:33.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.274+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1780365147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 msgr2=0x7fc9a0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.274+0000 7fc9a4e0c640 1 --2- 172.21.15.92:0/1780365147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc990009880 tx=0x7fc99002a5d0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.274+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1780365147 shutdown_connections 2024-09-11T22:05:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.274+0000 7fc9a4e0c640 1 --2- 172.21.15.92:0/1780365147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.274+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1780365147 >> 172.21.15.92:0/1780365147 conn(0x7fc9a00fc1b0 msgr2=0x7fc9a00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.274+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1780365147 shutdown_connections 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.274+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1780365147 wait complete. 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.275+0000 7fc9a4e0c640 1 Processor -- start 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.275+0000 7fc9a4e0c640 1 -- start start 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.275+0000 7fc9a4e0c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.275+0000 7fc9a4e0c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010aa40 con 0x7fc9a0106560 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.275+0000 7fc99e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.275+0000 7fc99e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35104/0 (socket says 172.21.15.92:35104) 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.275+0000 7fc99e59c640 1 -- 172.21.15.92:0/1059480893 learned_addr learned my addr 172.21.15.92:0/1059480893 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.276+0000 7fc99e59c640 1 -- 172.21.15.92:0/1059480893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a0197590 con 0x7fc9a0106560 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.276+0000 7fc99e59c640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0197050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc990009230 tx=0x7fc9900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.276+0000 7fc997fff640 1 -- 172.21.15.92:0/1059480893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc99003f070 con 0x7fc9a0106560 2024-09-11T22:05:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.276+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a0197820 con 0x7fc9a0106560 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.276+0000 7fc997fff640 1 -- 172.21.15.92:0/1059480893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc990004d10 con 0x7fc9a0106560 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.276+0000 7fc997fff640 1 -- 172.21.15.92:0/1059480893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc990035030 con 0x7fc9a0106560 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.276+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a019b920 con 0x7fc9a0106560 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.277+0000 7fc997fff640 1 -- 172.21.15.92:0/1059480893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc990042070 con 0x7fc9a0106560 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.277+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a0101bd0 con 0x7fc9a0106560 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.277+0000 7fc997fff640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc98c039060 0x7fc98c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.277+0000 7fc997fff640 1 -- 172.21.15.92:0/1059480893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc99006cf50 con 0x7fc9a0106560 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.277+0000 7fc995d9b640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc98c039060 0x7fc98c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.277+0000 7fc995d9b640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc98c039060 0x7fc98c03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098c0 tx=0x7fc988006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.282+0000 7fc997fff640 1 -- 172.21.15.92:0/1059480893 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc990039350 con 0x7fc9a0106560 2024-09-11T22:05:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.455+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9a01983b0 con 0x7fc98c039060 2024-09-11T22:05:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.456+0000 7fc997fff640 1 -- 172.21.15.92:0/1059480893 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fc9a01983b0 con 0x7fc98c039060 2024-09-11T22:05:33.457 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:33.457 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}] 2024-09-11T22:05:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc98c039060 msgr2=0x7fc98c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc98c039060 0x7fc98c03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098c0 tx=0x7fc988006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 msgr2=0x7fc9a0197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0197050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc990009230 tx=0x7fc9900048f0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 shutdown_connections 2024-09-11T22:05:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc98c039060 0x7fc98c03b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 --2- 172.21.15.92:0/1059480893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a0106560 0x7fc9a0197050 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.458+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 >> 172.21.15.92:0/1059480893 conn(0x7fc9a00fc1b0 msgr2=0x7fc9a0069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.459+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 shutdown_connections 2024-09-11T22:05:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:33.459+0000 7fc9a4e0c640 1 -- 172.21.15.92:0/1059480893 wait complete. 2024-09-11T22:05:34.034 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi153 2024-09-11T22:05:34.035 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:05:34.035 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.conf 2024-09-11T22:05:34.053 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:05:34.054 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:05:34.112 INFO:tasks.cephadm:Adding host smithi153 to orchestrator... 2024-09-11T22:05:34.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch host add smithi153 2024-09-11T22:05:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:34.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:34.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:34 smithi092 ceph-mon[31173]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:05:34.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.833+0000 7f08352f7640 1 -- 172.21.15.92:0/1334590963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 msgr2=0x7f0830106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:34.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.833+0000 7f08352f7640 1 --2- 172.21.15.92:0/1334590963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830106920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0820009810 tx=0x7f082002a540 comp rx=0 tx=0).stop 2024-09-11T22:05:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.833+0000 7f08352f7640 1 -- 172.21.15.92:0/1334590963 shutdown_connections 2024-09-11T22:05:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.833+0000 7f08352f7640 1 --2- 172.21.15.92:0/1334590963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830106920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.833+0000 7f08352f7640 1 -- 172.21.15.92:0/1334590963 >> 172.21.15.92:0/1334590963 conn(0x7f0830077310 msgr2=0x7f08300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.833+0000 7f08352f7640 1 -- 172.21.15.92:0/1334590963 shutdown_connections 2024-09-11T22:05:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.833+0000 7f08352f7640 1 -- 172.21.15.92:0/1334590963 wait complete. 2024-09-11T22:05:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f08352f7640 1 Processor -- start 2024-09-11T22:05:34.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f08352f7640 1 -- start start 2024-09-11T22:05:34.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f08352f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:34.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f08352f7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f083010aa20 con 0x7f0830106540 2024-09-11T22:05:34.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f082effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:34.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f082effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35116/0 (socket says 172.21.15.92:35116) 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f082effd640 1 -- 172.21.15.92:0/707335377 learned_addr learned my addr 172.21.15.92:0/707335377 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f082effd640 1 -- 172.21.15.92:0/707335377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08301974e0 con 0x7f0830106540 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.834+0000 7f082effd640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830196fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0820030b00 tx=0x7f0820030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.835+0000 7f082cff9640 1 -- 172.21.15.92:0/707335377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0820040070 con 0x7f0830106540 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.835+0000 7f082cff9640 1 -- 172.21.15.92:0/707335377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08200318b0 con 0x7f0830106540 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.835+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0830197770 con 0x7f0830106540 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.835+0000 7f082cff9640 1 -- 172.21.15.92:0/707335377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0820003de0 con 0x7f0830106540 2024-09-11T22:05:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.835+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083019b870 con 0x7f0830106540 2024-09-11T22:05:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.836+0000 7f082cff9640 1 -- 172.21.15.92:0/707335377 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0820039040 con 0x7f0830106540 2024-09-11T22:05:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.836+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0830101bb0 con 0x7f0830106540 2024-09-11T22:05:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.836+0000 7f082cff9640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f081c039060 0x7f081c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.836+0000 7f082cff9640 1 -- 172.21.15.92:0/707335377 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f082006d5c0 con 0x7f0830106540 2024-09-11T22:05:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.836+0000 7f08267fc640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f081c039060 0x7f081c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.836+0000 7f08267fc640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f081c039060 0x7f081c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f08180098c0 tx=0x7f0818006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:34.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:34.840+0000 7f082cff9640 1 -- 172.21.15.92:0/707335377 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0820043bd0 con 0x7f0830106540 2024-09-11T22:05:35.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:35.010+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}) -- 0x7f0830106920 con 0x7f081c039060 2024-09-11T22:05:37.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:37.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:37.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:37.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:37.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:37.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:37.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:05:37.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:37.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:05:38.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:37 smithi092 ceph-mon[31173]: Deploying cephadm binary to smithi153 2024-09-11T22:05:38.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:37 smithi092 ceph-mon[31173]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T22:05:38.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:37 smithi092 ceph-mon[31173]: Updating smithi110:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:05:38.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:37 smithi092 ceph-mon[31173]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:05:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.152+0000 7f082cff9640 1 -- 172.21.15.92:0/707335377 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0830106920 con 0x7f081c039060 2024-09-11T22:05:38.153 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi153' with addr '172.21.15.153' 2024-09-11T22:05:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f081c039060 msgr2=0x7f081c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f081c039060 0x7f081c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f08180098c0 tx=0x7f0818006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 msgr2=0x7f0830196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830196fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0820030b00 tx=0x7f0820030b30 comp rx=0 tx=0).stop 2024-09-11T22:05:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 shutdown_connections 2024-09-11T22:05:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f081c039060 0x7f081c03b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:38.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 --2- 172.21.15.92:0/707335377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0830106540 0x7f0830196fa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:38.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.154+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 >> 172.21.15.92:0/707335377 conn(0x7f0830077310 msgr2=0x7f08300696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:38.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.155+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 shutdown_connections 2024-09-11T22:05:38.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:38.155+0000 7f08352f7640 1 -- 172.21.15.92:0/707335377 wait complete. 2024-09-11T22:05:38.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T22:05:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:39.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: Updating smithi110:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:05:39.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:39.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:39.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:39.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:39.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:39.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:39.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:39.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:39.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 -- 172.21.15.92:0/3231158326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 msgr2=0x7f0c44102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 --2- 172.21.15.92:0/3231158326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44102b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009880 tx=0x7f0c3402a670 comp rx=0 tx=0).stop 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 -- 172.21.15.92:0/3231158326 shutdown_connections 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 --2- 172.21.15.92:0/3231158326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44102b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 -- 172.21.15.92:0/3231158326 >> 172.21.15.92:0/3231158326 conn(0x7f0c440fc1b0 msgr2=0x7f0c440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 -- 172.21.15.92:0/3231158326 shutdown_connections 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 -- 172.21.15.92:0/3231158326 wait complete. 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.405+0000 7f0c4a49d640 1 Processor -- start 2024-09-11T22:05:39.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c4a49d640 1 -- start start 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c4a49d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c4a49d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4410bb70 con 0x7f0c44102750 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36490/0 (socket says 172.21.15.92:36490) 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c43fff640 1 -- 172.21.15.92:0/2695046043 learned_addr learned my addr 172.21.15.92:0/2695046043 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c43fff640 1 -- 172.21.15.92:0/2695046043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c44197640 con 0x7f0c44102750 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.406+0000 7f0c43fff640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44197100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009230 tx=0x7f0c340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.407+0000 7f0c417fa640 1 -- 172.21.15.92:0/2695046043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3403f070 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.407+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c441978d0 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.407+0000 7f0c417fa640 1 -- 172.21.15.92:0/2695046043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c34004d10 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.407+0000 7f0c417fa640 1 -- 172.21.15.92:0/2695046043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34035030 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.407+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c44197d30 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.408+0000 7f0c417fa640 1 -- 172.21.15.92:0/2695046043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0c34042070 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.408+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4406a7b0 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.408+0000 7f0c417fa640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0c2c039060 0x7f0c2c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.408+0000 7f0c417fa640 1 -- 172.21.15.92:0/2695046043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0c3406cf20 con 0x7f0c44102750 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.408+0000 7f0c437fe640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0c2c039060 0x7f0c2c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.408+0000 7f0c437fe640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0c2c039060 0x7f0c2c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c380098c0 tx=0x7f0c38006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.413+0000 7f0c417fa640 1 -- 172.21.15.92:0/2695046043 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c34039350 con 0x7f0c44102750 2024-09-11T22:05:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.584+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c44198460 con 0x7f0c2c039060 2024-09-11T22:05:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.585+0000 7f0c417fa640 1 -- 172.21.15.92:0/2695046043 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f0c44198460 con 0x7f0c2c039060 2024-09-11T22:05:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:05:39.586 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}] 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.587+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0c2c039060 msgr2=0x7f0c2c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.587+0000 7f0c4a49d640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0c2c039060 0x7f0c2c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c380098c0 tx=0x7f0c38006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.587+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 msgr2=0x7f0c44197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.587+0000 7f0c4a49d640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44197100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009230 tx=0x7f0c340048f0 comp rx=0 tx=0).stop 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.588+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 shutdown_connections 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.588+0000 7f0c4a49d640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0c2c039060 0x7f0c2c03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.588+0000 7f0c4a49d640 1 --2- 172.21.15.92:0/2695046043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c44102750 0x7f0c44197100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.588+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 >> 172.21.15.92:0/2695046043 conn(0x7f0c440fc1b0 msgr2=0x7f0c44103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.588+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 shutdown_connections 2024-09-11T22:05:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:39.588+0000 7f0c4a49d640 1 -- 172.21.15.92:0/2695046043 wait complete. 2024-09-11T22:05:40.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:39 smithi092 ceph-mon[31173]: Added host smithi153 2024-09-11T22:05:40.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:39 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:40.231 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T22:05:40.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T22:05:40.468 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:40.865 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:40 smithi092 ceph-mon[31173]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:05:40.865 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:40 smithi092 ceph-mon[31173]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.944+0000 7f6990a6e640 1 -- 172.21.15.92:0/4209144844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 msgr2=0x7f698c102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.944+0000 7f6990a6e640 1 --2- 172.21.15.92:0/4209144844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c102bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f697c009c20 tx=0x7f697c02ab20 comp rx=0 tx=0).stop 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.944+0000 7f6990a6e640 1 -- 172.21.15.92:0/4209144844 shutdown_connections 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.944+0000 7f6990a6e640 1 --2- 172.21.15.92:0/4209144844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c102bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.944+0000 7f6990a6e640 1 -- 172.21.15.92:0/4209144844 >> 172.21.15.92:0/4209144844 conn(0x7f698c0fc1b0 msgr2=0x7f698c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.945+0000 7f6990a6e640 1 -- 172.21.15.92:0/4209144844 shutdown_connections 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.945+0000 7f6990a6e640 1 -- 172.21.15.92:0/4209144844 wait complete. 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.945+0000 7f6990a6e640 1 Processor -- start 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.945+0000 7f6990a6e640 1 -- start start 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f6990a6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f6990a6e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10bb70 con 0x7f698c1027f0 2024-09-11T22:05:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f698a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f698a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c100450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36500/0 (socket says 172.21.15.92:36500) 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f698a59c640 1 -- 172.21.15.92:0/2085937401 learned_addr learned my addr 172.21.15.92:0/2085937401 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f698a59c640 1 -- 172.21.15.92:0/2085937401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c102340 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f698a59c640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c100450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f697c005e30 tx=0x7f697c02e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f697b7fe640 1 -- 172.21.15.92:0/2085937401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697c047070 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f697b7fe640 1 -- 172.21.15.92:0/2085937401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697c0368b0 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c100990 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f697b7fe640 1 -- 172.21.15.92:0/2085937401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697c004660 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.946+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c100e50 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.947+0000 7f697b7fe640 1 -- 172.21.15.92:0/2085937401 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f697c04a070 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.947+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c06a710 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.947+0000 7f697b7fe640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6974039060 0x7f697403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.947+0000 7f697b7fe640 1 -- 172.21.15.92:0/2085937401 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f697c03d030 con 0x7f698c1027f0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.947+0000 7f6989d9b640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6974039060 0x7f697403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.948+0000 7f6989d9b640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6974039060 0x7f697403b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f69800098c0 tx=0x7f6980006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:40.952+0000 7f697b7fe640 1 -- 172.21.15.92:0/2085937401 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f697c032030 con 0x7f698c1027f0 2024-09-11T22:05:41.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.121+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f698c101980 con 0x7f698c1027f0 2024-09-11T22:05:41.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.746+0000 7f697b7fe640 1 -- 172.21.15.92:0/2085937401 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f697c02c5b0 con 0x7f698c1027f0 2024-09-11T22:05:41.746 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6974039060 msgr2=0x7f697403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6974039060 0x7f697403b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f69800098c0 tx=0x7f6980006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 msgr2=0x7f698c100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c100450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f697c005e30 tx=0x7f697c02e890 comp rx=0 tx=0).stop 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 shutdown_connections 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6974039060 0x7f697403b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 --2- 172.21.15.92:0/2085937401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c1027f0 0x7f698c100450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.748+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 >> 172.21.15.92:0/2085937401 conn(0x7f698c0fc1b0 msgr2=0x7f698c1055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.749+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 shutdown_connections 2024-09-11T22:05:41.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:05:41.749+0000 7f6990a6e640 1 -- 172.21.15.92:0/2085937401 wait complete. 2024-09-11T22:05:42.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:41 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2085937401' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T22:05:42.325 INFO:tasks.cephadm:Adding mon.a on smithi092 2024-09-11T22:05:42.325 INFO:tasks.cephadm:Adding mon.b on smithi110 2024-09-11T22:05:42.325 INFO:tasks.cephadm:Adding mon.c on smithi153 2024-09-11T22:05:42.325 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi092:172.21.15.92=a;smithi110:172.21.15.110=b;smithi153:172.21.15.153=c' 2024-09-11T22:05:42.627 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T22:05:42.676 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T22:05:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2085937401' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:05:43.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.173+0000 7f9e5759e640 1 -- 172.21.15.153:0/4227094320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 msgr2=0x7f9e58100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.173+0000 7f9e5759e640 1 --2- 172.21.15.153:0/4227094320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e58100710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009880 tx=0x7f9e4802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.173+0000 7f9e5759e640 1 -- 172.21.15.153:0/4227094320 shutdown_connections 2024-09-11T22:05:43.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.173+0000 7f9e5759e640 1 --2- 172.21.15.153:0/4227094320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e58100710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.173+0000 7f9e5759e640 1 -- 172.21.15.153:0/4227094320 >> 172.21.15.153:0/4227094320 conn(0x7f9e580fbaa0 msgr2=0x7f9e580fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.173+0000 7f9e5759e640 1 -- 172.21.15.153:0/4227094320 shutdown_connections 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.174+0000 7f9e5759e640 1 -- 172.21.15.153:0/4227094320 wait complete. 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.174+0000 7f9e5759e640 1 Processor -- start 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.174+0000 7f9e5759e640 1 -- start start 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.174+0000 7f9e5759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e580fd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.174+0000 7f9e5759e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581062d0 con 0x7f9e58100310 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.175+0000 7f9e5659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e580fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.175+0000 7f9e5659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e580fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.153:49824/0 (socket says 172.21.15.153:49824) 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.175+0000 7f9e5659c640 1 -- 172.21.15.153:0/2841459386 learned_addr learned my addr 172.21.15.153:0/2841459386 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.175+0000 7f9e5659c640 1 -- 172.21.15.153:0/2841459386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e580fdea0 con 0x7f9e58100310 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.175+0000 7f9e5659c640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e580fd960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009230 tx=0x7f9e480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.175+0000 7f9e4f7fe640 1 -- 172.21.15.153:0/2841459386 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e4803f070 con 0x7f9e58100310 2024-09-11T22:05:43.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.176+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e580fc090 con 0x7f9e58100310 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.176+0000 7f9e4f7fe640 1 -- 172.21.15.153:0/2841459386 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e48003aa0 con 0x7f9e58100310 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.176+0000 7f9e4f7fe640 1 -- 172.21.15.153:0/2841459386 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e48035030 con 0x7f9e58100310 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.176+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e580fc5d0 con 0x7f9e58100310 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.177+0000 7f9e4f7fe640 1 -- 172.21.15.153:0/2841459386 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9e48042070 con 0x7f9e58100310 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.177+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e58106a50 con 0x7f9e58100310 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.177+0000 7f9e4f7fe640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9e44039060 0x7f9e4403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.177+0000 7f9e4f7fe640 1 -- 172.21.15.153:0/2841459386 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9e4806d0f0 con 0x7f9e58100310 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.177+0000 7f9e4ffff640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9e44039060 0x7f9e4403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.178+0000 7f9e4ffff640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9e44039060 0x7f9e4403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e400098c0 tx=0x7f9e40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.182+0000 7f9e4f7fe640 1 -- 172.21.15.153:0/2841459386 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e4802f300 con 0x7f9e58100310 2024-09-11T22:05:43.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.364+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi110:172.21.15.110=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}) -- 0x7f9e5818fe60 con 0x7f9e44039060 2024-09-11T22:05:43.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.369+0000 7f9e4f7fe640 1 -- 172.21.15.153:0/2841459386 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9e5818fe60 con 0x7f9e44039060 2024-09-11T22:05:43.369 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mon update... 2024-09-11T22:05:43.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9e44039060 msgr2=0x7f9e4403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9e44039060 0x7f9e4403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e400098c0 tx=0x7f9e40006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 msgr2=0x7f9e580fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e580fd960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009230 tx=0x7f9e480048f0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 shutdown_connections 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9e44039060 0x7f9e4403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 --2- 172.21.15.153:0/2841459386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e58100310 0x7f9e580fd960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 >> 172.21.15.153:0/2841459386 conn(0x7f9e580fbaa0 msgr2=0x7f9e58069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 shutdown_connections 2024-09-11T22:05:43.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:43.371+0000 7f9e5759e640 1 -- 172.21.15.153:0/2841459386 wait complete. 2024-09-11T22:05:43.927 DEBUG:teuthology.orchestra.run.smithi110:mon.b> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T22:05:43.930 DEBUG:teuthology.orchestra.run.smithi153:mon.c> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T22:05:43.932 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:05:43.932 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:05:44.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: Updating smithi153:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:05:44.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:44 smithi092 ceph-mon[31173]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:05:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:44 smithi092 ceph-mon[31173]: Updating smithi153:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:05:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:44 smithi092 ceph-mon[31173]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi110:172.21.15.110=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:05:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:44 smithi092 ceph-mon[31173]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi110:172.21.15.110=b;smithi153:172.21.15.153=c;count:3 2024-09-11T22:05:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:44 smithi092 ceph-mon[31173]: Deploying daemon mon.c on smithi153 2024-09-11T22:05:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:44 smithi092 ceph-mon[31173]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.009+0000 7fb106320640 1 -- 172.21.15.153:0/4020013844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 msgr2=0x7fb100106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.009+0000 7fb106320640 1 --2- 172.21.15.153:0/4020013844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100106940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009880 tx=0x7fb0f002a5d0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.009+0000 7fb106320640 1 -- 172.21.15.153:0/4020013844 shutdown_connections 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.009+0000 7fb106320640 1 --2- 172.21.15.153:0/4020013844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100106940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.009+0000 7fb106320640 1 -- 172.21.15.153:0/4020013844 >> 172.21.15.153:0/4020013844 conn(0x7fb1000fc1b0 msgr2=0x7fb1000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.009+0000 7fb106320640 1 -- 172.21.15.153:0/4020013844 shutdown_connections 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.009+0000 7fb106320640 1 -- 172.21.15.153:0/4020013844 wait complete. 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.010+0000 7fb106320640 1 Processor -- start 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.010+0000 7fb106320640 1 -- start start 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.010+0000 7fb106320640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.010+0000 7fb106320640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb10010aa40 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.153:49846/0 (socket says 172.21.15.153:49846) 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fffff640 1 -- 172.21.15.153:0/3067150585 learned_addr learned my addr 172.21.15.153:0/3067150585 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fffff640 1 -- 172.21.15.153:0/3067150585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb100198350 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fffff640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100197e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fdffb640 1 -- 172.21.15.153:0/3067150585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0f003f070 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1001985e0 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fdffb640 1 -- 172.21.15.153:0/3067150585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0f0003aa0 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb0fdffb640 1 -- 172.21.15.153:0/3067150585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0f0035030 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.011+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb100198a40 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.012+0000 7fb0fdffb640 1 -- 172.21.15.153:0/3067150585 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb0f0042070 con 0x7fb100106560 2024-09-11T22:05:46.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.012+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb100101bd0 con 0x7fb100106560 2024-09-11T22:05:46.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.012+0000 7fb0fdffb640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb0ec039060 0x7fb0ec03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.013+0000 7fb0fdffb640 1 -- 172.21.15.153:0/3067150585 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb0f006cd80 con 0x7fb100106560 2024-09-11T22:05:46.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.013+0000 7fb0f7fff640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb0ec039060 0x7fb0ec03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.013+0000 7fb0f7fff640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb0ec039060 0x7fb0ec03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80098c0 tx=0x7fb0e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.017+0000 7fb0fdffb640 1 -- 172.21.15.153:0/3067150585 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0f004a260 con 0x7fb100106560 2024-09-11T22:05:46.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.253+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb1001069c0 con 0x7fb100106560 2024-09-11T22:05:46.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.253+0000 7fb0fdffb640 1 -- 172.21.15.153:0/3067150585 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb0f0031400 con 0x7fb100106560 2024-09-11T22:05:46.254 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:05:46.254 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":1,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","modified":"2024-09-11T22:04:27.004735Z","created":"2024-09-11T22:04:27.004735Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T22:05:46.255 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 1 2024-09-11T22:05:46.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb0ec039060 msgr2=0x7fb0ec03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb0ec039060 0x7fb0ec03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80098c0 tx=0x7fb0e8006db0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 msgr2=0x7fb100197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100197e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f00048f0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 shutdown_connections 2024-09-11T22:05:46.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb0ec039060 0x7fb0ec03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 --2- 172.21.15.153:0/3067150585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb100106560 0x7fb100197e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.255+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 >> 172.21.15.153:0/3067150585 conn(0x7fb1000fc1b0 msgr2=0x7fb1000fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:46.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.256+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 shutdown_connections 2024-09-11T22:05:46.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:46.256+0000 7fb106320640 1 -- 172.21.15.153:0/3067150585 wait complete. 2024-09-11T22:05:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:46 smithi092 ceph-mon[31173]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:46 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/3067150585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:05:48.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:48 smithi153 systemd[1]: Started Ceph mon.c for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:05:48.314 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:05:48.314 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:05:48.580 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:05:53.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.438+0000 7fa8c7932640 1 -- 172.21.15.153:0/1700839365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8ac003530 msgr2=0x7fa8ac0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.438+0000 7fa8c7932640 1 --2- 172.21.15.153:0/1700839365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8ac003530 0x7fa8ac0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0102af0 tx=0x7fa8b402c490 comp rx=0 tx=0).stop 2024-09-11T22:05:53.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.438+0000 7fa8c7932640 1 -- 172.21.15.153:0/1700839365 shutdown_connections 2024-09-11T22:05:53.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.438+0000 7fa8c7932640 1 --2- 172.21.15.153:0/1700839365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8ac003530 0x7fa8ac0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.438+0000 7fa8c7932640 1 -- 172.21.15.153:0/1700839365 >> 172.21.15.153:0/1700839365 conn(0x7fa8c00fd320 msgr2=0x7fa8c00ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.438+0000 7fa8c7932640 1 -- 172.21.15.153:0/1700839365 shutdown_connections 2024-09-11T22:05:53.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.438+0000 7fa8c7932640 1 -- 172.21.15.153:0/1700839365 wait complete. 2024-09-11T22:05:53.440 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:05:53.440 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: mon.a calling monitor election 2024-09-11T22:05:53.440 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.440 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.440 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:53.440 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: mon.c calling monitor election 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: monmap epoch 2 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: last_changed 2024-09-11T22:05:48.091145+0000 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: created 2024-09-11T22:04:27.004735+0000 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: min_mon_release 19 (squid) 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: election_strategy: 1 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-11T22:05:53.441 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: fsmap 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: mgrmap e13: x(active, since 37s) 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: overall HEALTH_OK 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:05:53.442 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.439+0000 7fa8c7932640 1 Processor -- start 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c7932640 1 -- start start 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c7932640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 0x7fa8c0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c7932640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c01999d0 0x7fa8c019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c7932640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0109630 con 0x7fa8c01999d0 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c7932640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01094b0 con 0x7fa8ac003530 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c4ecd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c01999d0 0x7fa8c019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c4ecd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c01999d0 0x7fa8c019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.153:58292/0 (socket says 172.21.15.153:58292) 2024-09-11T22:05:53.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c4ecd640 1 -- 172.21.15.153:0/1893243232 learned_addr learned my addr 172.21.15.153:0/1893243232 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.440+0000 7fa8c56ce640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 0x7fa8c0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c56ce640 1 -- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 msgr2=0x7fa8c0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c56ce640 1 -- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 msgr2=0x7fa8c0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c56ce640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 0x7fa8c0199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c56ce640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 0x7fa8c0199490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c4ecd640 1 -- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 msgr2=0x7fa8c0199490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c4ecd640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 0x7fa8c0199490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c4ecd640 1 -- 172.21.15.153:0/1893243232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c0199ff0 con 0x7fa8c01999d0 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c4ecd640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c01999d0 0x7fa8c019dd60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b4a0 tx=0x7fa8bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8b27fc640 1 -- 172.21.15.153:0/1893243232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa8bc010610 con 0x7fa8c01999d0 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8b27fc640 1 -- 172.21.15.153:0/1893243232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8bc010c60 con 0x7fa8c01999d0 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c019e2a0 con 0x7fa8c01999d0 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8b27fc640 1 -- 172.21.15.153:0/1893243232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa8bc017da0 con 0x7fa8c01999d0 2024-09-11T22:05:53.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.441+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c019e7e0 con 0x7fa8c01999d0 2024-09-11T22:05:53.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.442+0000 7fa8b27fc640 1 -- 172.21.15.153:0/1893243232 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa8bc017070 con 0x7fa8c01999d0 2024-09-11T22:05:53.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.443+0000 7fa8b27fc640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa8a4038e50 0x7fa8a403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.443+0000 7fa8b27fc640 1 -- 172.21.15.153:0/1893243232 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa8bc00d040 con 0x7fa8c01999d0 2024-09-11T22:05:53.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.443+0000 7fa8c56ce640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa8a4038e50 0x7fa8a403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.443+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c0102da0 con 0x7fa8c01999d0 2024-09-11T22:05:53.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.444+0000 7fa8c56ce640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa8a4038e50 0x7fa8a403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4012c10 tx=0x7fa8b40323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.448+0000 7fa8b27fc640 1 -- 172.21.15.153:0/1893243232 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8bc01e020 con 0x7fa8c01999d0 2024-09-11T22:05:53.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: mon.a calling monitor election 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: mon.c calling monitor election 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: monmap epoch 2 2024-09-11T22:05:53.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: last_changed 2024-09-11T22:05:48.091145+0000 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: created 2024-09-11T22:04:27.004735+0000 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: min_mon_release 19 (squid) 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: election_strategy: 1 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: fsmap 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: mgrmap e13: x(active, since 37s) 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: overall HEALTH_OK 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:05:53.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:05:53.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:53.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.681+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa8c019eb80 con 0x7fa8c01999d0 2024-09-11T22:05:53.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.682+0000 7fa8b27fc640 1 -- 172.21.15.153:0/1893243232 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa8bc020b80 con 0x7fa8c01999d0 2024-09-11T22:05:53.683 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:05:53.683 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":2,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","modified":"2024-09-11T22:05:48.091145Z","created":"2024-09-11T22:04:27.004735Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T22:05:53.683 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 2 2024-09-11T22:05:53.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa8a4038e50 msgr2=0x7fa8a403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa8a4038e50 0x7fa8a403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4012c10 tx=0x7fa8b40323a0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c01999d0 msgr2=0x7fa8c019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c01999d0 0x7fa8c019dd60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b4a0 tx=0x7fa8bc00b960 comp rx=0 tx=0).stop 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 shutdown_connections 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa8a4038e50 0x7fa8a403b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c01999d0 0x7fa8c019dd60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 --2- 172.21.15.153:0/1893243232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac003530 0x7fa8c0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 >> 172.21.15.153:0/1893243232 conn(0x7fa8c00fd320 msgr2=0x7fa8c00fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 shutdown_connections 2024-09-11T22:05:53.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:53.684+0000 7fa8c7932640 1 -- 172.21.15.153:0/1893243232 wait complete. 2024-09-11T22:05:54.307 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:54 smithi153 ceph-mon[35045]: Deploying daemon mon.b on smithi110 2024-09-11T22:05:54.308 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:54 smithi153 ceph-mon[35045]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:54.308 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:54 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/1893243232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:05:54.308 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:54.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:54 smithi092 ceph-mon[31173]: Deploying daemon mon.b on smithi110 2024-09-11T22:05:54.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:54 smithi092 ceph-mon[31173]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:54 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/1893243232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:05:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:05:55.337 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:05:55.337 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:05:55.586 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:05:55.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 -- 172.21.15.153:0/438901532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c107660 msgr2=0x7f552c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 --2- 172.21.15.153:0/438901532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c107660 0x7f552c107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f551c009880 tx=0x7f551c02a5d0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 -- 172.21.15.153:0/438901532 shutdown_connections 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 --2- 172.21.15.153:0/438901532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c107660 0x7f552c107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 -- 172.21.15.153:0/438901532 >> 172.21.15.153:0/438901532 conn(0x7f552c0fd320 msgr2=0x7f552c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 -- 172.21.15.153:0/438901532 shutdown_connections 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 -- 172.21.15.153:0/438901532 wait complete. 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.989+0000 7f553253f640 1 Processor -- start 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f553253f640 1 -- start start 2024-09-11T22:05:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f553253f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 0x7f552c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f553253f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c197780 0x7f552c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f553253f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f552c10bbd0 con 0x7f552c197780 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f553253f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f552c10ba50 con 0x7f552c107660 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 0x7f552c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 0x7f552c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57158/0 (socket says 172.21.15.153:57158) 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552bfff640 1 -- 172.21.15.153:0/2946211017 learned_addr learned my addr 172.21.15.153:0/2946211017 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552b7fe640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c197780 0x7f552c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552bfff640 1 -- 172.21.15.153:0/2946211017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 msgr2=0x7f552c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T22:05:55.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552bfff640 1 -- 172.21.15.153:0/2946211017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 msgr2=0x7f552c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552bfff640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 0x7f552c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.990+0000 7f552bfff640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 0x7f552c197240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f552b7fe640 1 -- 172.21.15.153:0/2946211017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 msgr2=0x7f552c197240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f552b7fe640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 0x7f552c197240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f552b7fe640 1 -- 172.21.15.153:0/2946211017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c197da0 con 0x7f552c197780 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f552b7fe640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c197780 0x7f552c19bb10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f552000b3d0 tx=0x7f552000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f55297fa640 1 -- 172.21.15.153:0/2946211017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5520011630 con 0x7f552c197780 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f55297fa640 1 -- 172.21.15.153:0/2946211017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5520011c80 con 0x7f552c197780 2024-09-11T22:05:55.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c19c050 con 0x7f552c197780 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f55297fa640 1 -- 172.21.15.153:0/2946211017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5520010d80 con 0x7f552c197780 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.991+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c19c590 con 0x7f552c197780 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.992+0000 7f55297fa640 1 -- 172.21.15.153:0/2946211017 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5520018020 con 0x7f552c197780 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.992+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c102da0 con 0x7f552c197780 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.992+0000 7f55297fa640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5514038df0 0x7f551403b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.992+0000 7f55297fa640 1 -- 172.21.15.153:0/2946211017 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f552004bd30 con 0x7f552c197780 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.992+0000 7f552bfff640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5514038df0 0x7f551403b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.993+0000 7f552bfff640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5514038df0 0x7f551403b2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f551c02abc0 tx=0x7f551c02aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:55.997+0000 7f55297fa640 1 -- 172.21.15.153:0/2946211017 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f552001f020 con 0x7f552c197780 2024-09-11T22:05:56.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.223+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f552c19c8a0 con 0x7f552c197780 2024-09-11T22:05:56.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.224+0000 7f55297fa640 1 -- 172.21.15.153:0/2946211017 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f5520018790 con 0x7f552c197780 2024-09-11T22:05:56.225 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:05:56.225 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":2,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","modified":"2024-09-11T22:05:48.091145Z","created":"2024-09-11T22:04:27.004735Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T22:05:56.225 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 2 2024-09-11T22:05:56.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5514038df0 msgr2=0x7f551403b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:56.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5514038df0 0x7f551403b2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f551c02abc0 tx=0x7f551c02aad0 comp rx=0 tx=0).stop 2024-09-11T22:05:56.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c197780 msgr2=0x7f552c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:56.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c197780 0x7f552c19bb10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f552000b3d0 tx=0x7f552000b890 comp rx=0 tx=0).stop 2024-09-11T22:05:56.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 shutdown_connections 2024-09-11T22:05:56.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f5514038df0 0x7f551403b2b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:56.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c197780 0x7f552c19bb10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:56.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 --2- 172.21.15.153:0/2946211017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f552c107660 0x7f552c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:56.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.226+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 >> 172.21.15.153:0/2946211017 conn(0x7f552c0fd320 msgr2=0x7f552c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:56.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.227+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 shutdown_connections 2024-09-11T22:05:56.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:05:56.227+0000 7f553253f640 1 -- 172.21.15.153:0/2946211017 wait complete. 2024-09-11T22:05:57.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:56 smithi092 ceph-mon[31173]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:57.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:05:56 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/2946211017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:05:57.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:56 smithi153 ceph-mon[35045]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:05:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:05:56 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/2946211017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:05:57.224 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:05:57 smithi110 systemd[1]: Started Ceph mon.b for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:05:57.808 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:05:57.808 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:05:58.061 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:06:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.081+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/968738027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b0005740 msgr2=0x7fa2b0007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.081+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/968738027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b0005740 0x7fa2b0007bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2c41016c0 tx=0x7fa2c002c3f0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.082+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/968738027 shutdown_connections 2024-09-11T22:06:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.082+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/968738027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b0005740 0x7fa2b0007bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.082+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/968738027 >> 172.21.15.153:0/968738027 conn(0x7fa2c40fc280 msgr2=0x7fa2c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.082+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/968738027 shutdown_connections 2024-09-11T22:06:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.082+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/968738027 wait complete. 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.083+0000 7fa2cbc0d640 1 Processor -- start 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.083+0000 7fa2cbc0d640 1 -- start start 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2cbc0d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 0x7fa2c4195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2cbc0d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c4195590 0x7fa2c4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2cbc0d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 0x7fa2c419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2cbc0d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4107570 con 0x7fa2c4195590 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2cbc0d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41073f0 con 0x7fa2b0005740 2024-09-11T22:06:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2cbc0d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41076f0 con 0x7fa2c4199e60 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2ca1aa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 0x7fa2c419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2ca1aa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 0x7fa2c419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57172/0 (socket says 172.21.15.153:57172) 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2ca1aa640 1 -- 172.21.15.153:0/3432761773 learned_addr learned my addr 172.21.15.153:0/3432761773 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c99a9640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 0x7fa2c4195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2ca1aa640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 msgr2=0x7fa2c419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2ca1aa640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 msgr2=0x7fa2c419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2ca1aa640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 0x7fa2c419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c91a8640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c4195590 0x7fa2c4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c99a9640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 msgr2=0x7fa2c4195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c99a9640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 msgr2=0x7fa2c4195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c99a9640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 0x7fa2c4195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:06:02.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2ca1aa640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 0x7fa2c419a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c99a9640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 0x7fa2c4195050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c91a8640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 msgr2=0x7fa2c419a310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c91a8640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 0x7fa2c419a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c91a8640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 msgr2=0x7fa2c4195050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c91a8640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 0x7fa2c4195050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.084+0000 7fa2c91a8640 1 -- 172.21.15.153:0/3432761773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c41a1bd0 con 0x7fa2c4195590 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.085+0000 7fa2c91a8640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c4195590 0x7fa2c4199920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b450 tx=0x7fa2b400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.085+0000 7fa2baffd640 1 -- 172.21.15.153:0/3432761773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4004230 con 0x7fa2c4195590 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.085+0000 7fa2baffd640 1 -- 172.21.15.153:0/3432761773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b40043d0 con 0x7fa2c4195590 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.085+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c41a1e00 con 0x7fa2c4195590 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.085+0000 7fa2baffd640 1 -- 172.21.15.153:0/3432761773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4003be0 con 0x7fa2c4195590 2024-09-11T22:06:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.085+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c41a2380 con 0x7fa2c4195590 2024-09-11T22:06:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.086+0000 7fa2baffd640 1 -- 172.21.15.153:0/3432761773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa2b400f050 con 0x7fa2c4195590 2024-09-11T22:06:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.086+0000 7fa2baffd640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2a80394f0 0x7fa2a803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.086+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c41084d0 con 0x7fa2c4195590 2024-09-11T22:06:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.086+0000 7fa2baffd640 1 -- 172.21.15.153:0/3432761773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa2b404bde0 con 0x7fa2c4195590 2024-09-11T22:06:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.086+0000 7fa2c99a9640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2a80394f0 0x7fa2a803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.087+0000 7fa2c99a9640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2a80394f0 0x7fa2a803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c00149c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.091+0000 7fa2baffd640 1 -- 172.21.15.153:0/3432761773 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2b400d040 con 0x7fa2c4195590 2024-09-11T22:06:02.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.333+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa2c402d7c0 con 0x7fa2c4195590 2024-09-11T22:06:02.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.333+0000 7fa2baffd640 1 -- 172.21.15.153:0/3432761773 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7fa2b4021bf0 con 0x7fa2c4195590 2024-09-11T22:06:02.334 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:06:02.334 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":3,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","modified":"2024-09-11T22:05:56.976347Z","created":"2024-09-11T22:04:27.004735Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T22:06:02.334 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 3 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: mon.a calling monitor election 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: mon.c calling monitor election 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: mon.b calling monitor election 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.335 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: monmap epoch 3 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: last_changed 2024-09-11T22:05:56.976347+0000 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: created 2024-09-11T22:04:27.004735+0000 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: min_mon_release 19 (squid) 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: election_strategy: 1 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: fsmap 2024-09-11T22:06:02.336 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:06:02.337 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: mgrmap e13: x(active, since 46s) 2024-09-11T22:06:02.337 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: overall HEALTH_OK 2024-09-11T22:06:02.337 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.337 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.337 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.337 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2a80394f0 msgr2=0x7fa2a803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2a80394f0 0x7fa2a803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c00149c0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c4195590 msgr2=0x7fa2c4199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c4195590 0x7fa2c4199920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b450 tx=0x7fa2b400b910 comp rx=0 tx=0).stop 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 shutdown_connections 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2a80394f0 0x7fa2a803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2c4199e60 0x7fa2c419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c4195590 0x7fa2c4199920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 --2- 172.21.15.153:0/3432761773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0005740 0x7fa2c4195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 >> 172.21.15.153:0/3432761773 conn(0x7fa2c40fc280 msgr2=0x7fa2c4105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 shutdown_connections 2024-09-11T22:06:02.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:02.336+0000 7fa2cbc0d640 1 -- 172.21.15.153:0/3432761773 wait complete. 2024-09-11T22:06:02.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:06:02.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: mon.a calling monitor election 2024-09-11T22:06:02.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: mon.c calling monitor election 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: mon.b calling monitor election 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: monmap epoch 3 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:06:02.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: last_changed 2024-09-11T22:05:56.976347+0000 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: created 2024-09-11T22:04:27.004735+0000 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: min_mon_release 19 (squid) 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: election_strategy: 1 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: fsmap 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: mgrmap e13: x(active, since 46s) 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: overall HEALTH_OK 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: mon.a calling monitor election 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: mon.c calling monitor election 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: mon.b calling monitor election 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: monmap epoch 3 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: last_changed 2024-09-11T22:05:56.976347+0000 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: created 2024-09-11T22:04:27.004735+0000 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: min_mon_release 19 (squid) 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: election_strategy: 1 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-11T22:06:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: fsmap 2024-09-11T22:06:02.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:06:02.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: mgrmap e13: x(active, since 46s) 2024-09-11T22:06:02.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: overall HEALTH_OK 2024-09-11T22:06:02.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:02.931 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T22:06:02.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T22:06:03.182 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:03 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:03.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:03 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/3432761773' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:06:03.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:03 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:03 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:03 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/3432761773' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:06:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:03 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:03.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:03 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:03.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:03 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/3432761773' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:06:03.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:03 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:06:03.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 -- 172.21.15.92:0/601598414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c106540 msgr2=0x7f1e4c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:03.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 --2- 172.21.15.92:0/601598414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c106540 0x7f1e4c106920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009810 tx=0x7f1e4402a540 comp rx=0 tx=0).stop 2024-09-11T22:06:03.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 -- 172.21.15.92:0/601598414 shutdown_connections 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 --2- 172.21.15.92:0/601598414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c106540 0x7f1e4c106920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 -- 172.21.15.92:0/601598414 >> 172.21.15.92:0/601598414 conn(0x7f1e4c077310 msgr2=0x7f1e4c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 -- 172.21.15.92:0/601598414 shutdown_connections 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 -- 172.21.15.92:0/601598414 wait complete. 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.581+0000 7f1e51168640 1 Processor -- start 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e51168640 1 -- start start 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e51168640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c106540 0x7f1e4c1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e51168640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4c196710 0x7f1e4c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e51168640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c19afe0 0x7f1e4c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e51168640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10a9d0 con 0x7f1e4c19afe0 2024-09-11T22:06:03.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e51168640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10a850 con 0x7f1e4c106540 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e51168640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10ab50 con 0x7f1e4c196710 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c19afe0 0x7f1e4c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c19afe0 0x7f1e4c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59566/0 (socket says 172.21.15.92:59566) 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 -- 172.21.15.92:0/1426926398 learned_addr learned my addr 172.21.15.92:0/1426926398 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4ad9d640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c106540 0x7f1e4c1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4a59c640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4c196710 0x7f1e4c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 -- 172.21.15.92:0/1426926398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4c196710 msgr2=0x7f1e4c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4c196710 0x7f1e4c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 -- 172.21.15.92:0/1426926398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c106540 msgr2=0x7f1e4c1961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c106540 0x7f1e4c1961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 -- 172.21.15.92:0/1426926398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c1a2d50 con 0x7f1e4c19afe0 2024-09-11T22:06:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4a59c640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4c196710 0x7f1e4c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:03.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.582+0000 7f1e4b59e640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c19afe0 0x7f1e4c19b490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00b450 tx=0x7f1e3c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:03.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.583+0000 7f1e33fff640 1 -- 172.21.15.92:0/1426926398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00f900 con 0x7f1e4c19afe0 2024-09-11T22:06:03.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.583+0000 7f1e33fff640 1 -- 172.21.15.92:0/1426926398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e3c017420 con 0x7f1e4c19afe0 2024-09-11T22:06:03.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.583+0000 7f1e33fff640 1 -- 172.21.15.92:0/1426926398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00e660 con 0x7f1e4c19afe0 2024-09-11T22:06:03.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.583+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c069710 con 0x7f1e4c19afe0 2024-09-11T22:06:03.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.583+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c069c80 con 0x7f1e4c19afe0 2024-09-11T22:06:03.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.584+0000 7f1e33fff640 1 -- 172.21.15.92:0/1426926398 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1e3c00e800 con 0x7f1e4c19afe0 2024-09-11T22:06:03.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.584+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c101bb0 con 0x7f1e4c19afe0 2024-09-11T22:06:03.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.584+0000 7f1e33fff640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1e340394f0 0x7f1e3403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:03.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.584+0000 7f1e33fff640 1 -- 172.21.15.92:0/1426926398 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1e3c04da80 con 0x7f1e4c19afe0 2024-09-11T22:06:03.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.584+0000 7f1e4ad9d640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1e340394f0 0x7f1e3403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:03.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.584+0000 7f1e4ad9d640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1e340394f0 0x7f1e3403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1e440092c0 tx=0x7f1e44009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:03.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.589+0000 7f1e33fff640 1 -- 172.21.15.92:0/1426926398 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e3c00fc30 con 0x7f1e4c19afe0 2024-09-11T22:06:03.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.759+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1e4c069f70 con 0x7f1e4c19afe0 2024-09-11T22:06:03.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.760+0000 7f1e33fff640 1 -- 172.21.15.92:0/1426926398 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f1e3c016bf0 con 0x7f1e4c19afe0 2024-09-11T22:06:03.761 INFO:teuthology.orchestra.run.smithi092.stdout:# minimal ceph.conf for a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:06:03.761 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-11T22:06:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:06:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: mon_host = [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1e340394f0 msgr2=0x7f1e3403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1e340394f0 0x7f1e3403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1e440092c0 tx=0x7f1e44009250 comp rx=0 tx=0).stop 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c19afe0 msgr2=0x7f1e4c19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c19afe0 0x7f1e4c19b490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00b450 tx=0x7f1e3c00b910 comp rx=0 tx=0).stop 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 shutdown_connections 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1e340394f0 0x7f1e3403b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e4c19afe0 0x7f1e4c19b490 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4c196710 0x7f1e4c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:03.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 --2- 172.21.15.92:0/1426926398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c106540 0x7f1e4c1961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:03.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 >> 172.21.15.92:0/1426926398 conn(0x7f1e4c077310 msgr2=0x7f1e4c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:03.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 shutdown_connections 2024-09-11T22:06:03.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:03.762+0000 7f1e51168640 1 -- 172.21.15.92:0/1426926398 wait complete. 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:04.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T22:06:04.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T22:06:04.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-11T22:06:04.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:04 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1426926398' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-11T22:06:04.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:04 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1426926398' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:04.485 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T22:06:04.485 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:06:04.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:06:04.511 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:06:04.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T22:06:04.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-11T22:06:04.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:04 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1426926398' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:04.576 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:06:04.576 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:06:04.604 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:06:04.604 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:06:04.669 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:06:04.670 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:06:04.697 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:06:04.697 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:06:04.762 INFO:tasks.cephadm:Adding mgr.x on smithi092 2024-09-11T22:06:04.762 INFO:tasks.cephadm:Adding mgr.y on smithi110 2024-09-11T22:06:04.762 INFO:tasks.cephadm:Adding mgr.z on smithi153 2024-09-11T22:06:04.763 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi092=x;smithi110=y;smithi153=z' 2024-09-11T22:06:05.036 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:06:05.452 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: Updating smithi092:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.452 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: Updating smithi153:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.452 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: Updating smithi110:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.452 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.452 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.452 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:05.453 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:05 smithi153 ceph-mon[35045]: Reconfiguring daemon mon.a on smithi092 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 -- 172.21.15.153:0/922063785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 msgr2=0x7f9cbc101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/922063785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 0x7f9cbc101ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009880 tx=0x7f9cb802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 -- 172.21.15.153:0/922063785 shutdown_connections 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/922063785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 0x7f9cbc101ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 -- 172.21.15.153:0/922063785 >> 172.21.15.153:0/922063785 conn(0x7f9cbc0fce90 msgr2=0x7f9cbc0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 -- 172.21.15.153:0/922063785 shutdown_connections 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 -- 172.21.15.153:0/922063785 wait complete. 2024-09-11T22:06:05.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 Processor -- start 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.465+0000 7f9cc40bf640 1 -- start start 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc40bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 0x7f9cbc196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc40bf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cbc197440 0x7f9cbc19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc40bf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cbc1978c0 0x7f9cbc19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc40bf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc107680 con 0x7f9cbc1016e0 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc40bf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc107500 con 0x7f9cbc1978c0 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc40bf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc107800 con 0x7f9cbc197440 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cbc197440 0x7f9cbc19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cbc197440 0x7f9cbc19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49458/0 (socket says 172.21.15.153:49458) 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 -- 172.21.15.153:0/1751627075 learned_addr learned my addr 172.21.15.153:0/1751627075 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc1e5b640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 0x7f9cbc196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc265c640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cbc1978c0 0x7f9cbc19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:05.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 -- 172.21.15.153:0/1751627075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cbc1978c0 msgr2=0x7f9cbc19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cbc1978c0 0x7f9cbc19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 -- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 msgr2=0x7f9cbc196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 0x7f9cbc196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 -- 172.21.15.153:0/1751627075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc19f300 con 0x7f9cbc197440 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc1e5b640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 0x7f9cbc196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cc165a640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cbc197440 0x7f9cbc19c4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00b520 tx=0x7f9cac00b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.466+0000 7f9cb2ffd640 1 -- 172.21.15.153:0/1751627075 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac004230 con 0x7f9cbc197440 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.467+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc19f5f0 con 0x7f9cbc197440 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.467+0000 7f9cb2ffd640 1 -- 172.21.15.153:0/1751627075 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cac0043d0 con 0x7f9cbc197440 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.467+0000 7f9cb2ffd640 1 -- 172.21.15.153:0/1751627075 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac003cf0 con 0x7f9cbc197440 2024-09-11T22:06:05.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.467+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc19fbb0 con 0x7f9cbc197440 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: Updating smithi092:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: Updating smithi153:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: Updating smithi110:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:05.471 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:05 smithi110 ceph-mon[34830]: Reconfiguring daemon mon.a on smithi092 2024-09-11T22:06:05.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.467+0000 7f9cb2ffd640 1 -- 172.21.15.153:0/1751627075 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9cac00e070 con 0x7f9cbc197440 2024-09-11T22:06:05.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.468+0000 7f9cb2ffd640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9ca80394f0 0x7f9ca803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:05.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.468+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cbc190440 con 0x7f9cbc197440 2024-09-11T22:06:05.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.468+0000 7f9cb2ffd640 1 -- 172.21.15.153:0/1751627075 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9cac04bc10 con 0x7f9cbc197440 2024-09-11T22:06:05.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.468+0000 7f9cc1e5b640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9ca80394f0 0x7f9ca803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:05.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.468+0000 7f9cc1e5b640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9ca80394f0 0x7f9ca803b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80314e0 tx=0x7f9cb8032eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:05.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.473+0000 7f9cb2ffd640 1 -- 172.21.15.153:0/1751627075 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9cac014020 con 0x7f9cbc197440 2024-09-11T22:06:05.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: Updating smithi092:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: Updating smithi153:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: Updating smithi110:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T22:06:05.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:05.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:05.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:05.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:05 smithi092 ceph-mon[31173]: Reconfiguring daemon mon.a on smithi092 2024-09-11T22:06:05.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.649+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi110=y;smithi153=z", "target": ["mon-mgr", ""]}) -- 0x7f9cbc0626c0 con 0x7f9ca80394f0 2024-09-11T22:06:05.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.653+0000 7f9cb2ffd640 1 -- 172.21.15.153:0/1751627075 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9cbc0626c0 con 0x7f9ca80394f0 2024-09-11T22:06:05.654 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mgr update... 2024-09-11T22:06:05.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9ca80394f0 msgr2=0x7f9ca803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9ca80394f0 0x7f9ca803b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80314e0 tx=0x7f9cb8032eb0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cbc197440 msgr2=0x7f9cbc19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cbc197440 0x7f9cbc19c4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00b520 tx=0x7f9cac00b9e0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 shutdown_connections 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9ca80394f0 0x7f9ca803b9b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cbc1978c0 0x7f9cbc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cbc197440 0x7f9cbc19c4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 --2- 172.21.15.153:0/1751627075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cbc1016e0 0x7f9cbc196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 >> 172.21.15.153:0/1751627075 conn(0x7f9cbc0fce90 msgr2=0x7f9cbc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 shutdown_connections 2024-09-11T22:06:05.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:06:05.656+0000 7f9cc40bf640 1 -- 172.21.15.153:0/1751627075 wait complete. 2024-09-11T22:06:06.384 DEBUG:teuthology.orchestra.run.smithi110:mgr.y> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T22:06:06.387 DEBUG:teuthology.orchestra.run.smithi153:mgr.z> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T22:06:06.390 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T22:06:06.390 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:06:06.390 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:06:06.410 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:06:06.410 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-11T22:06:06.468 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:06:06.468 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:06.468 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-11T22:06:06.468 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:06.469 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:06.469 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 22:05:25.743241729 +0000 2024-09-11T22:06:06.469 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 22:00:27.449436853 +0000 2024-09-11T22:06:06.469 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 22:00:27.449436853 +0000 2024-09-11T22:06:06.469 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 22:00:27.449436853 +0000 2024-09-11T22:06:06.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:06:06.538 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T22:06:06.538 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T22:06:06.538 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000235999 s, 2.2 MB/s 2024-09-11T22:06:06.540 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:06:06.599 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 22:05:25.773241405 +0000 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 22:00:27.775433505 +0000 2024-09-11T22:06:06.658 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 22:00:27.775433505 +0000 2024-09-11T22:06:06.659 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 22:00:27.775433505 +0000 2024-09-11T22:06:06.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: Reconfiguring daemon mon.b on smithi110 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi110=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:06.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: Saving service mgr spec with placement smithi092=x;smithi110=y;smithi153=z;count:3 2024-09-11T22:06:06.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.729 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T22:06:06.729 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T22:06:06.730 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000233355 s, 2.2 MB/s 2024-09-11T22:06:06.731 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: Reconfiguring daemon mon.b on smithi110 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi110=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: Saving service mgr spec with placement smithi092=x;smithi110=y;smithi153=z;count:3 2024-09-11T22:06:06.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.788 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: Reconfiguring daemon mon.b on smithi110 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi110=y;smithi153=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:06.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: Saving service mgr spec with placement smithi092=x;smithi110=y;smithi153=z;count:3 2024-09-11T22:06:06.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:06.847 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:06:06.847 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:06.847 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T22:06:06.847 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:06.847 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:06.847 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 22:05:25.803241083 +0000 2024-09-11T22:06:06.848 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 22:00:28.118429983 +0000 2024-09-11T22:06:06.848 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 22:00:28.118429983 +0000 2024-09-11T22:06:06.848 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 22:00:28.118429983 +0000 2024-09-11T22:06:06.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:06:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T22:06:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T22:06:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000289295 s, 1.8 MB/s 2024-09-11T22:06:06.916 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:06:06.974 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-11T22:06:07.031 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 22:05:25.833240760 +0000 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 22:00:28.459426482 +0000 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 22:00:28.459426482 +0000 2024-09-11T22:06:07.032 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 22:00:28.459426482 +0000 2024-09-11T22:06:07.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:06:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T22:06:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T22:06:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00024449 s, 2.1 MB/s 2024-09-11T22:06:07.099 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:06:07.156 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T22:06:07.157 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:06:07.173 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:06:07.173 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 22:05:56.430956796 +0000 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 22:00:33.131547049 +0000 2024-09-11T22:06:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 22:00:33.131547049 +0000 2024-09-11T22:06:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 22:00:33.131547049 +0000 2024-09-11T22:06:07.230 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:06:07.293 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T22:06:07.294 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T22:06:07.294 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000115412 s, 4.4 MB/s 2024-09-11T22:06:07.295 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:06:07.352 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-11T22:06:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:06:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:07.408 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-11T22:06:07.408 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:07.408 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:07.408 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 22:05:56.430956796 +0000 2024-09-11T22:06:07.408 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 22:00:33.467540900 +0000 2024-09-11T22:06:07.408 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 22:00:33.467540900 +0000 2024-09-11T22:06:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 22:00:33.467540900 +0000 2024-09-11T22:06:07.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:06:07.471 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T22:06:07.471 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T22:06:07.471 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000110256 s, 4.6 MB/s 2024-09-11T22:06:07.472 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:06:07.528 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-11T22:06:07.583 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:06:07.583 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:07.583 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T22:06:07.584 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:07.584 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:07.584 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 22:05:56.431956777 +0000 2024-09-11T22:06:07.584 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 22:00:33.775535264 +0000 2024-09-11T22:06:07.584 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 22:00:33.775535264 +0000 2024-09-11T22:06:07.584 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 22:00:33.775535264 +0000 2024-09-11T22:06:07.584 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:06:07.648 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T22:06:07.648 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T22:06:07.648 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000106239 s, 4.8 MB/s 2024-09-11T22:06:07.649 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:06:07.705 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-11T22:06:07.761 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:06:07.761 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:07.761 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-11T22:06:07.761 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:07.761 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:07.761 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 22:05:56.431956777 +0000 2024-09-11T22:06:07.761 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 22:00:34.099529335 +0000 2024-09-11T22:06:07.762 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 22:00:34.099529335 +0000 2024-09-11T22:06:07.762 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 22:00:34.099529335 +0000 2024-09-11T22:06:07.762 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:06:07.824 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T22:06:07.825 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T22:06:07.825 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000121909 s, 4.2 MB/s 2024-09-11T22:06:07.826 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:06:07.882 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-11T22:06:07.882 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:06:07.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:06:07.900 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-09-11T22:06:07.956 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-11 22:05:47.525432093 +0000 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-11 22:00:30.386351059 +0000 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-11 22:00:30.386351059 +0000 2024-09-11T22:06:07.957 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-11 22:00:30.386351059 +0000 2024-09-11T22:06:07.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:06:07.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:07.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:07.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:07 smithi110 ceph-mon[34830]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:06:07.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:07.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:07.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:07.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:07 smithi110 ceph-mon[34830]: Reconfiguring daemon mon.c on smithi153 2024-09-11T22:06:07.987 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:07.987 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:07.987 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:07 smithi153 ceph-mon[35045]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:06:07.987 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:07.987 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:07.987 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:07.987 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:07 smithi153 ceph-mon[35045]: Reconfiguring daemon mon.c on smithi153 2024-09-11T22:06:08.012 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-11T22:06:08.012 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-11T22:06:08.012 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.00025728 s, 2.0 MB/s 2024-09-11T22:06:08.013 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:06:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:07 smithi092 ceph-mon[31173]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:06:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:06:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:06:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:07 smithi092 ceph-mon[31173]: Reconfiguring daemon mon.c on smithi153 2024-09-11T22:06:08.069 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-09-11T22:06:08.124 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:06:08.124 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:08.124 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T22:06:08.124 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:08.125 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:08.125 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-11 22:05:47.526432075 +0000 2024-09-11T22:06:08.125 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-11 22:00:30.731344851 +0000 2024-09-11T22:06:08.125 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-11 22:00:30.731344851 +0000 2024-09-11T22:06:08.125 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-11 22:00:30.731344851 +0000 2024-09-11T22:06:08.125 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:06:08.188 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-11T22:06:08.188 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-11T22:06:08.188 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000186848 s, 2.7 MB/s 2024-09-11T22:06:08.189 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:06:08.245 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-09-11T22:06:08.301 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:06:08.301 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:08.301 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T22:06:08.302 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:08.302 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:08.302 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-11 22:05:47.526432075 +0000 2024-09-11T22:06:08.302 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-11 22:00:31.044339218 +0000 2024-09-11T22:06:08.302 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-11 22:00:31.044339218 +0000 2024-09-11T22:06:08.302 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-11 22:00:31.044339218 +0000 2024-09-11T22:06:08.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:06:08.368 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-11T22:06:08.368 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-11T22:06:08.368 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000178181 s, 2.9 MB/s 2024-09-11T22:06:08.369 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:06:08.426 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-09-11T22:06:08.481 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-11 22:05:47.527432056 +0000 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-11 22:00:31.356333603 +0000 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-11 22:00:31.356333603 +0000 2024-09-11T22:06:08.482 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-11 22:00:31.356333603 +0000 2024-09-11T22:06:08.482 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:06:08.545 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-11T22:06:08.546 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-11T22:06:08.546 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000215577 s, 2.4 MB/s 2024-09-11T22:06:08.547 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:06:08.605 INFO:tasks.cephadm:Deploying osd.0 on smithi092 with /dev/vg_nvme/lv_4... 2024-09-11T22:06:08.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:06:08.854 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:08.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:06:08.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:08.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:08.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:08.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:08.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:08.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:08.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:06:08.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:08.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:09.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:09 smithi110 ceph-mon[34830]: Deploying daemon mgr.z on smithi153 2024-09-11T22:06:09.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:09 smithi092 ceph-mon[31173]: Deploying daemon mgr.z on smithi153 2024-09-11T22:06:10.017 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:09 smithi153 ceph-mon[35045]: Deploying daemon mgr.z on smithi153 2024-09-11T22:06:10.018 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:09 smithi153 systemd[1]: Starting Ceph mgr.z for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:06:10.289 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 podman[36826]: 2024-09-11 22:06:09.91989587 +0000 UTC m=+0.015586046 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:06:10.289 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 podman[36826]: 2024-09-11 22:06:10.045454806 +0000 UTC m=+0.141144980 container create eb4416760230fe3e3571946b8b74ea2b5e66c2f29cb1d7c6d0152e76e6cd0713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T22:06:10.289 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 podman[36826]: 2024-09-11 22:06:10.231448506 +0000 UTC m=+0.327138682 container init eb4416760230fe3e3571946b8b74ea2b5e66c2f29cb1d7c6d0152e76e6cd0713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:06:10.289 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 podman[36826]: 2024-09-11 22:06:10.234277366 +0000 UTC m=+0.329967542 container start eb4416760230fe3e3571946b8b74ea2b5e66c2f29cb1d7c6d0152e76e6cd0713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T22:06:10.290 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mgr[36846]: -- 172.21.15.153:0/2776543106 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a828a30e0 con 0x556a82880800 2024-09-11T22:06:10.573 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 bash[36826]: eb4416760230fe3e3571946b8b74ea2b5e66c2f29cb1d7c6d0152e76e6cd0713 2024-09-11T22:06:10.573 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 systemd[1]: Started Ceph mgr.z for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:06:10.573 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:10.358+0000 7fe3fc19f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:06:10.573 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:10.422+0000 7fe3fc19f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:06:10.573 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:10.572+0000 7fe3fc19f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:06:10.932 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:10.639+0000 7fe3fc19f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:06:10.932 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:10.790+0000 7fe3fc19f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:06:10.932 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:10.930+0000 7fe3fc19f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:06:10.932 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:10.932 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.932 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.933 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.933 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:10.933 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:06:10.933 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:10.933 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:06:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:10.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:10.984 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:10.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:10.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:10.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:06:10.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:10.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:06:11.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-09-11T22:06:11.416 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:11.538 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:11.250+0000 7fe3fc19f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:06:11.539 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:11.310+0000 7fe3fc19f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:06:11.539 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:11.495+0000 7fe3fc19f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:06:11.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:11 smithi092 ceph-mon[31173]: Deploying daemon mgr.y on smithi110 2024-09-11T22:06:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.802+0000 7fbac56a2640 1 -- 172.21.15.92:0/1264779610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 msgr2=0x7fbac0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.802+0000 7fbac56a2640 1 --2- 172.21.15.92:0/1264779610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0100ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009900 tx=0x7fbab002a410 comp rx=0 tx=0).stop 2024-09-11T22:06:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 -- 172.21.15.92:0/1264779610 shutdown_connections 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 --2- 172.21.15.92:0/1264779610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbac0102690 0x7fbac0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 --2- 172.21.15.92:0/1264779610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbac0101cd0 0x7fbac0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 --2- 172.21.15.92:0/1264779610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0100ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 -- 172.21.15.92:0/1264779610 >> 172.21.15.92:0/1264779610 conn(0x7fbac00fc280 msgr2=0x7fbac00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 -- 172.21.15.92:0/1264779610 shutdown_connections 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 -- 172.21.15.92:0/1264779610 wait complete. 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 Processor -- start 2024-09-11T22:06:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.803+0000 7fbac56a2640 1 -- start start 2024-09-11T22:06:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbac56a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbac56a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbac0101cd0 0x7fbac01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbac56a2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbac0102690 0x7fbac01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbac56a2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010c6d0 con 0x7fbac0100ad0 2024-09-11T22:06:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbac56a2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010c550 con 0x7fbac0101cd0 2024-09-11T22:06:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbac56a2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010c850 con 0x7fbac0102690 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52110/0 (socket says 172.21.15.92:52110) 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 -- 172.21.15.92:0/3072020928 learned_addr learned my addr 172.21.15.92:0/3072020928 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbab67fc640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbac0101cd0 0x7fbac01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabf7fe640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbac0102690 0x7fbac01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 -- 172.21.15.92:0/3072020928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbac0102690 msgr2=0x7fbac01035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbac0102690 0x7fbac01035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 -- 172.21.15.92:0/3072020928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbac0101cd0 msgr2=0x7fbac01030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbac0101cd0 0x7fbac01030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 -- 172.21.15.92:0/3072020928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac0103eb0 con 0x7fbac0100ad0 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabeffd640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0104a60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbab00314e0 tx=0x7fbab0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab003f070 con 0x7fbac0100ad0 2024-09-11T22:06:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab0031e40 con 0x7fbac0100ad0 2024-09-11T22:06:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0035030 con 0x7fbac0100ad0 2024-09-11T22:06:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.804+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac01a2080 con 0x7fbac0100ad0 2024-09-11T22:06:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.805+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01a2560 con 0x7fbac0100ad0 2024-09-11T22:06:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.805+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbab0042070 con 0x7fbac0100ad0 2024-09-11T22:06:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.805+0000 7fbabcff9640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbaac039290 0x7fbaac03b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.805+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac01a2790 con 0x7fbac0100ad0 2024-09-11T22:06:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.806+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbab006cf80 con 0x7fbac0100ad0 2024-09-11T22:06:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.806+0000 7fbab67fc640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbaac039290 0x7fbaac03b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.806+0000 7fbab67fc640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbaac039290 0x7fbaac03b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbac01047e0 tx=0x7fbaa8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.810+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbab00351d0 con 0x7fbac0100ad0 2024-09-11T22:06:11.885 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:11 smithi110 ceph-mon[34830]: Deploying daemon mgr.y on smithi110 2024-09-11T22:06:11.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:11.985+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbac006a710 con 0x7fbaac039290 2024-09-11T22:06:12.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:11 smithi153 ceph-mon[35045]: Deploying daemon mgr.y on smithi110 2024-09-11T22:06:12.217 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 systemd[1]: Starting Ceph mgr.y for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:06:12.534 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 podman[36178]: 2024-09-11 22:06:12.172384849 +0000 UTC m=+0.016228266 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:06:12.534 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 podman[36178]: 2024-09-11 22:06:12.324076182 +0000 UTC m=+0.167919599 container create ab1b1fe2c08bb6529f65fb2d2f3f2f1cba392dea38175190e1a679fb5d897781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-11T22:06:12.535 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 podman[36178]: 2024-09-11 22:06:12.534844441 +0000 UTC m=+0.378687871 container init ab1b1fe2c08bb6529f65fb2d2f3f2f1cba392dea38175190e1a679fb5d897781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T22:06:12.711 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:12.391+0000 7fe3fc19f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:06:12.711 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:12.458+0000 7fe3fc19f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:06:12.874 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:12.874 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 podman[36178]: 2024-09-11 22:06:12.538262662 +0000 UTC m=+0.382106080 container start ab1b1fe2c08bb6529f65fb2d2f3f2f1cba392dea38175190e1a679fb5d897781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-mgr[36197]: -- 172.21.15.110:0/2194582737 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b0c8eb0e0 con 0x558b0c8c9000 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 bash[36178]: ab1b1fe2c08bb6529f65fb2d2f3f2f1cba392dea38175190e1a679fb5d897781 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 systemd[1]: Started Ceph mgr.y for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:06:12.875 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:12.658+0000 7f17c9726040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:06:12.876 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:12.718+0000 7f17c9726040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:06:12.876 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:12.873+0000 7f17c9726040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:06:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:06:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:06:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:13.217 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:12.942+0000 7f17c9726040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:06:13.217 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:13 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:13.102+0000 7f17c9726040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:06:13.504 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:13.244+0000 7fe3fc19f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:06:13.504 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:13.364+0000 7fe3fc19f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:06:13.584 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:13 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:13.256+0000 7f17c9726040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:06:13.584 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:13 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:13.582+0000 7f17c9726040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:06:13.774 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:13 smithi092 ceph-mon[31173]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:13.774 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:13 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:13.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-mon[35045]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:13.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:13.789 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:13.517+0000 7fe3fc19f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:06:13.789 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:13.580+0000 7fe3fc19f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:06:13.789 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:13.734+0000 7fe3fc19f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:06:13.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:13 smithi110 ceph-mon[34830]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:13.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:13 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:13.967 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:13 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:13.644+0000 7f17c9726040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:06:13.967 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:13 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:13.831+0000 7f17c9726040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:06:14.108 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:13.804+0000 7fe3fc19f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:06:14.108 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:13.871+0000 7fe3fc19f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:06:14.108 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:14.039+0000 7fe3fc19f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:06:14.108 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:14.106+0000 7fe3fc19f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:06:14.380 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:14.174+0000 7fe3fc19f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:06:14.380 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:14.258+0000 7fe3fc19f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:06:14.380 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:14.317+0000 7fe3fc19f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:06:14.380 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:14.378+0000 7fe3fc19f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:14.713+0000 7f17c9726040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:14.780+0000 7f17c9726040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3009278448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4595b0bf-4ae7-499d-98e9-3aa88e41ff70"}]: dispatch 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3009278448' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4595b0bf-4ae7-499d-98e9-3aa88e41ff70"}]': finished 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:14.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:14 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/496970755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:06:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3009278448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4595b0bf-4ae7-499d-98e9-3aa88e41ff70"}]: dispatch 2024-09-11T22:06:15.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3009278448' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4595b0bf-4ae7-499d-98e9-3aa88e41ff70"}]': finished 2024-09-11T22:06:15.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:06:15.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:15.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:14 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/496970755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:06:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:06:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3009278448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4595b0bf-4ae7-499d-98e9-3aa88e41ff70"}]: dispatch 2024-09-11T22:06:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3009278448' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4595b0bf-4ae7-499d-98e9-3aa88e41ff70"}]': finished 2024-09-11T22:06:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:06:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:14 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/496970755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:06:15.538 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z[36842]: 2024-09-11T22:06:15.173+0000 7fe3fc19f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:06:15.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:15.722+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fbab003b560 con 0x7fbac0100ad0 2024-09-11T22:06:15.744 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:15.487+0000 7f17c9726040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:06:15.744 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:15.602+0000 7f17c9726040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:06:15.744 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:15.743+0000 7f17c9726040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: Reconfiguring daemon mgr.x on smithi092 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: Standby manager daemon z started 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:06:15.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:16.018 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:15.804+0000 7f17c9726040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:06:16.018 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:15 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:15.950+0000 7f17c9726040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:06:16.018 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.017+0000 7f17c9726040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:06:16.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:06:16.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: Reconfiguring daemon mgr.x on smithi092 2024-09-11T22:06:16.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: Standby manager daemon z started 2024-09-11T22:06:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:06:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:06:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:06:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:06:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: Reconfiguring daemon mgr.x on smithi092 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: Standby manager daemon z started 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.153:0/279740736' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:06:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:16.294 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.076+0000 7f17c9726040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:06:16.295 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.230+0000 7f17c9726040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:06:16.295 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.293+0000 7f17c9726040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:06:16.558 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.359+0000 7f17c9726040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:06:16.558 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.440+0000 7f17c9726040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:06:16.558 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.499+0000 7f17c9726040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:06:16.558 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:16.557+0000 7f17c9726040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:06:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-mon[34830]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-mon[34830]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-11T22:06:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:06:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:16 smithi092 ceph-mon[31173]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:16 smithi092 ceph-mon[31173]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-11T22:06:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:06:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:16 smithi153 ceph-mon[35045]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:16 smithi153 ceph-mon[35045]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-11T22:06:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:06:17.717 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:06:17 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y[36193]: 2024-09-11T22:06:17.336+0000 7f17c9726040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:06:17.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:17.730+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fbab004d3a0 con 0x7fbac0100ad0 2024-09-11T22:06:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:17 smithi092 ceph-mon[31173]: Standby manager daemon y started 2024-09-11T22:06:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:17 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:06:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:17 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:06:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:17 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:06:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:17 smithi092 ceph-mon[31173]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:06:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:17 smithi153 ceph-mon[35045]: Standby manager daemon y started 2024-09-11T22:06:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:17 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:06:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:17 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:06:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:17 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:06:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:17 smithi153 ceph-mon[35045]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:06:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:17 smithi110 ceph-mon[34830]: Standby manager daemon y started 2024-09-11T22:06:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:17 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:06:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:17 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:06:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:17 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:06:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:17 smithi110 ceph-mon[34830]: from='mgr.? 172.21.15.110:0/3212782346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:06:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:18 smithi092 ceph-mon[31173]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:18 smithi092 ceph-mon[31173]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-11T22:06:19.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:06:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:18 smithi153 ceph-mon[35045]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:18 smithi153 ceph-mon[35045]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-11T22:06:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:06:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:18 smithi110 ceph-mon[34830]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:18 smithi110 ceph-mon[34830]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-11T22:06:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:06:21.005 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:20 smithi092 ceph-mon[31173]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:21.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:20 smithi153 ceph-mon[35045]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:21.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:20 smithi110 ceph-mon[34830]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:22.151 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.151 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.151 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:22.151 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:22.151 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:22.152 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.152 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:21 smithi092 ceph-mon[31173]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:21 smithi110 ceph-mon[34830]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:21 smithi153 ceph-mon[35045]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:23.674 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:06:23.674 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:23.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:06:23.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:23.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:06:23.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:24.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:24 smithi092 ceph-mon[31173]: Deploying daemon osd.0 on smithi092 2024-09-11T22:06:24.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:24 smithi092 ceph-mon[31173]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:24.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:24 smithi110 ceph-mon[34830]: Deploying daemon osd.0 on smithi092 2024-09-11T22:06:24.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:24 smithi110 ceph-mon[34830]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:24.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:24 smithi153 ceph-mon[35045]: Deploying daemon osd.0 on smithi092 2024-09-11T22:06:24.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:24 smithi153 ceph-mon[35045]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:26.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:26 smithi110 ceph-mon[34830]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:27.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:26 smithi092 ceph-mon[31173]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:27.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:26 smithi153 ceph-mon[35045]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:28.935 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:28 smithi092 ceph-mon[31173]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:28.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:28 smithi110 ceph-mon[34830]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:29.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:28 smithi153 ceph-mon[35045]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:29.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:29.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:29.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:29.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:29.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:29.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:30.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:30.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:30.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:30.860 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:30 smithi092 ceph-mon[31173]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:30.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:30 smithi110 ceph-mon[34830]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:31.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:30 smithi153 ceph-mon[35045]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:32.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:32 smithi110 ceph-mon[34830]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:32.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:32.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:32.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:32.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:32.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:32.970 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:32 smithi092 ceph-mon[31173]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:32.970 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:32.970 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:32.970 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:32.970 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:32.971 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:32 smithi153 ceph-mon[35045]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:33.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.232+0000 7fbabcff9640 1 -- 172.21.15.92:0/3072020928 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbac006a710 con 0x7fbaac039290 2024-09-11T22:06:33.234 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 0 on host 'smithi092' 2024-09-11T22:06:33.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbaac039290 msgr2=0x7fbaac03b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbaac039290 0x7fbaac03b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbac01047e0 tx=0x7fbaa8009110 comp rx=0 tx=0).stop 2024-09-11T22:06:33.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 msgr2=0x7fbac0104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0104a60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbab00314e0 tx=0x7fbab0003d50 comp rx=0 tx=0).stop 2024-09-11T22:06:33.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 shutdown_connections 2024-09-11T22:06:33.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbaac039290 0x7fbaac03b750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbac0102690 0x7fbac01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbac0101cd0 0x7fbac01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 --2- 172.21.15.92:0/3072020928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0100ad0 0x7fbac0104a60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 >> 172.21.15.92:0/3072020928 conn(0x7fbac00fc280 msgr2=0x7fbac006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:33.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 shutdown_connections 2024-09-11T22:06:33.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:06:33.235+0000 7fbac56a2640 1 -- 172.21.15.92:0/3072020928 wait complete. 2024-09-11T22:06:33.866 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:33.866 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:33.866 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:33.892 DEBUG:teuthology.orchestra.run.smithi092:osd.0> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T22:06:33.895 INFO:tasks.cephadm:Deploying osd.1 on smithi110 with /dev/vg_nvme/lv_4... 2024-09-11T22:06:33.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:06:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:34.133 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:06:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:34 smithi110 ceph-mon[34830]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:34 smithi110 ceph-mon[34830]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:06:34.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:34 smithi092 ceph-mon[31173]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:34.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:34 smithi092 ceph-mon[31173]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:06:34.998 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:06:34 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-11T22:06:34.649+0000 7fbad8bc0640 -1 osd.0 0 log_to_monitors true 2024-09-11T22:06:35.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:34 smithi153 ceph-mon[35045]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:35.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:34 smithi153 ceph-mon[35045]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:06:36.420 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:06:36.449 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-09-11T22:06:36.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:36 smithi092 ceph-mon[31173]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:06:36.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:36 smithi092 ceph-mon[31173]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:06:36.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:36 smithi092 ceph-mon[31173]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:06:36.507 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:36.507 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:36 smithi092 ceph-mon[31173]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:36 smithi153 ceph-mon[35045]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:06:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:36 smithi153 ceph-mon[35045]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:06:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:36 smithi153 ceph-mon[35045]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:06:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:36 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:36 smithi153 ceph-mon[35045]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:36.711 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:06:36.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:36 smithi110 ceph-mon[34830]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:06:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:36 smithi110 ceph-mon[34830]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:06:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:36 smithi110 ceph-mon[34830]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:06:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:36 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:36 smithi110 ceph-mon[34830]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:36.768 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:06:36 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-11T22:06:36.742+0000 7fbad4b43640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T22:06:36.768 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:06:36 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-11T22:06:36.748+0000 7fbacc963640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:06:37.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 -- 172.21.15.110:0/3470296856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 msgr2=0x7ffacc107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 --2- 172.21.15.110:0/3470296856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc107400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009900 tx=0x7ffab402a410 comp rx=0 tx=0).stop 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 -- 172.21.15.110:0/3470296856 shutdown_connections 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 --2- 172.21.15.110:0/3470296856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffacc107940 0x7ffacc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 --2- 172.21.15.110:0/3470296856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc107400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 --2- 172.21.15.110:0/3470296856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffacc1026e0 0x7ffacc104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 -- 172.21.15.110:0/3470296856 >> 172.21.15.110:0/3470296856 conn(0x7ffacc0fc280 msgr2=0x7ffacc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 -- 172.21.15.110:0/3470296856 shutdown_connections 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.128+0000 7ffad2895640 1 -- 172.21.15.110:0/3470296856 wait complete. 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 Processor -- start 2024-09-11T22:06:37.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 -- start start 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffacc1026e0 0x7ffacc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffacc107940 0x7ffacc19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc10d780 con 0x7ffacc107940 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc10d600 con 0x7ffacc105010 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffad2895640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc10d900 con 0x7ffacc1026e0 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffacb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffacb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34486/0 (socket says 172.21.15.110:34486) 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.129+0000 7ffacb7fe640 1 -- 172.21.15.110:0/883787257 learned_addr learned my addr 172.21.15.110:0/883787257 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffacbfff640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffacc1026e0 0x7ffacc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffad0e32640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffacc107940 0x7ffacc19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffacb7fe640 1 -- 172.21.15.110:0/883787257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffacc1026e0 msgr2=0x7ffacc197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffacb7fe640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffacc1026e0 0x7ffacc197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffacb7fe640 1 -- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffacc107940 msgr2=0x7ffacc19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffacb7fe640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffacc107940 0x7ffacc19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffacb7fe640 1 -- 172.21.15.110:0/883787257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc19faf0 con 0x7ffacc105010 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffacb7fe640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc198400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffab40314e0 tx=0x7ffab4031a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffac97fa640 1 -- 172.21.15.110:0/883787257 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab403f030 con 0x7ffacc105010 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffac97fa640 1 -- 172.21.15.110:0/883787257 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffab40044b0 con 0x7ffacc105010 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffac97fa640 1 -- 172.21.15.110:0/883787257 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4035070 con 0x7ffacc105010 2024-09-11T22:06:37.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc19fd80 con 0x7ffacc105010 2024-09-11T22:06:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.130+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc1a0280 con 0x7ffacc105010 2024-09-11T22:06:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.131+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffacc0fdd70 con 0x7ffacc105010 2024-09-11T22:06:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.133+0000 7ffac97fa640 1 -- 172.21.15.110:0/883787257 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ffab4042030 con 0x7ffacc105010 2024-09-11T22:06:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.133+0000 7ffac97fa640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffab80a2b20 0x7ffab80a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.134+0000 7ffacbfff640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffab80a2b20 0x7ffab80a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.134+0000 7ffac97fa640 1 -- 172.21.15.110:0/883787257 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7ffab40ede00 con 0x7ffacc105010 2024-09-11T22:06:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.134+0000 7ffacbfff640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffab80a2b20 0x7ffab80a4fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009bd0 tx=0x7ffabc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.137+0000 7ffac97fa640 1 -- 172.21.15.110:0/883787257 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffab40bc3c0 con 0x7ffacc105010 2024-09-11T22:06:37.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:37.319+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ffacc062630 con 0x7ffab80a2b20 2024-09-11T22:06:37.320 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:06:37.320 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:06:37.320 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:37.320 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:37.320 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi092 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:37.321 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.322 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:37 smithi110 ceph-mon[34830]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' 2024-09-11T22:06:37.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi092 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:37.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:37 smithi092 ceph-mon[31173]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi092 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:06:37.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-11T22:06:37.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:37.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:37.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:37.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:37 smithi153 ceph-mon[35045]: from='osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998]' entity='osd.0' 2024-09-11T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:06:38.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:38.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:38.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] boot 2024-09-11T22:06:38.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:06:38.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:38.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] boot 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:06:38.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:38.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: osd.0 [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] boot 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:06:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/17381971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]: dispatch 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]: dispatch 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]': finished 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:40 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/1595766102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:06:40.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:06:40.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/17381971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]: dispatch 2024-09-11T22:06:40.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]: dispatch 2024-09-11T22:06:40.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]': finished 2024-09-11T22:06:40.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:06:40.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:06:40.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:40.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:40 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/1595766102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:06:40.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:06:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/17381971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]: dispatch 2024-09-11T22:06:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]: dispatch 2024-09-11T22:06:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26c935bd-8f25-4bbd-bdcf-8d55523c3e34"}]': finished 2024-09-11T22:06:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:06:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:06:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:40 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/1595766102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:06:42.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:42 smithi110 ceph-mon[34830]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:42 smithi092 ceph-mon[31173]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:43.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:42 smithi153 ceph-mon[35045]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:44.862 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:44 smithi110 ceph-mon[34830]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:44 smithi092 ceph-mon[31173]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:45.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:44 smithi153 ceph-mon[35045]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:46.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:46 smithi110 ceph-mon[34830]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:46 smithi092 ceph-mon[31173]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:47.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:46 smithi153 ceph-mon[35045]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:47.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:06:47.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:48.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:06:48.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:48.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:06:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:48.892 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:48 smithi110 ceph-mon[34830]: Deploying daemon osd.1 on smithi110 2024-09-11T22:06:48.893 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:48 smithi110 ceph-mon[34830]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:49.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:48 smithi092 ceph-mon[31173]: Deploying daemon osd.1 on smithi110 2024-09-11T22:06:49.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:48 smithi092 ceph-mon[31173]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:48 smithi153 ceph-mon[35045]: Deploying daemon osd.1 on smithi110 2024-09-11T22:06:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:48 smithi153 ceph-mon[35045]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:50.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:50 smithi110 ceph-mon[34830]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:50 smithi092 ceph-mon[31173]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:51.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:50 smithi153 ceph-mon[35045]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:52.901 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:52 smithi110 ceph-mon[34830]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:53.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:52 smithi092 ceph-mon[31173]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:53.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:52 smithi153 ceph-mon[35045]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:53.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:53 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:53.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:53 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:53.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:53 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:54.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:54.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:54.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:54.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:54.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:54.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:54.861 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:54 smithi110 ceph-mon[34830]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:55.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:54 smithi092 ceph-mon[31173]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:55.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:54 smithi153 ceph-mon[35045]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:56.959 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:56 smithi110 ceph-mon[34830]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:56.959 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:56 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:56.959 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:56 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:56.959 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:56 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:56.959 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:56 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:56.959 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:56 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:56 smithi092 ceph-mon[31173]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:57.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:56 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:56 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:56 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:57.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:56 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:57.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:56 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:56 smithi153 ceph-mon[35045]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:56 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:56 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:56 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:56 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:56 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.510+0000 7ffac97fa640 1 -- 172.21.15.110:0/883787257 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffacc062630 con 0x7ffab80a2b20 2024-09-11T22:06:57.512 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 1 on host 'smithi110' 2024-09-11T22:06:57.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.512+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffab80a2b20 msgr2=0x7ffab80a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:57.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.512+0000 7ffad2895640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffab80a2b20 0x7ffab80a4fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009bd0 tx=0x7ffabc005f90 comp rx=0 tx=0).stop 2024-09-11T22:06:57.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.512+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 msgr2=0x7ffacc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:57.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.512+0000 7ffad2895640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc198400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffab40314e0 tx=0x7ffab4031a30 comp rx=0 tx=0).stop 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 shutdown_connections 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffab80a2b20 0x7ffab80a4fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffacc107940 0x7ffacc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffacc105010 0x7ffacc198400 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 --2- 172.21.15.110:0/883787257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffacc1026e0 0x7ffacc197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 >> 172.21.15.110:0/883787257 conn(0x7ffacc0fc280 msgr2=0x7ffacc103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 shutdown_connections 2024-09-11T22:06:57.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:06:57.513+0000 7ffad2895640 1 -- 172.21.15.110:0/883787257 wait complete. 2024-09-11T22:06:57.808 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:57.808 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:57.808 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:58.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:58.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:58.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:58.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:06:58.126 DEBUG:teuthology.orchestra.run.smithi110:osd.1> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T22:06:58.129 INFO:tasks.cephadm:Deploying osd.2 on smithi153 with /dev/vg_nvme/lv_4... 2024-09-11T22:06:58.129 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:06:58.381 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:06:58.767 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:58 smithi153 ceph-mon[35045]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:58.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:58 smithi110 ceph-mon[34830]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:59.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:58 smithi092 ceph-mon[31173]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:06:59.569 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:06:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1[39273]: 2024-09-11T22:06:59.256+0000 7f4f56de6640 -1 osd.1 0 log_to_monitors true 2024-09-11T22:06:59.749 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:59 smithi153 ceph-mon[35045]: from='osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:06:59.749 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:06:59 smithi153 ceph-mon[35045]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:06:59.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:59 smithi110 ceph-mon[34830]: from='osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:06:59.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:06:59 smithi110 ceph-mon[34830]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:07:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:59 smithi092 ceph-mon[31173]: from='osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:07:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:06:59 smithi092 ceph-mon[31173]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:07:00.886 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:00 smithi153 ceph-mon[35045]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:07:00.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:00 smithi153 ceph-mon[35045]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:07:00.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:00 smithi153 ceph-mon[35045]: from='osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:07:00.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:00 smithi153 ceph-mon[35045]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:07:00.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:00.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:00 smithi153 ceph-mon[35045]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:07:00.890 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:07:00.920 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2024-09-11T22:07:00.950 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1[39273]: 2024-09-11T22:07:00.712+0000 7f4f52d69640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T22:07:00.950 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1[39273]: 2024-09-11T22:07:00.716+0000 7f4f4eb91640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:07:00.950 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-mon[34830]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:07:00.950 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-mon[34830]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:07:00.950 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-mon[34830]: from='osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:07:00.950 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-mon[34830]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:07:00.951 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:00.951 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:00 smithi110 ceph-mon[34830]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:07:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:00 smithi092 ceph-mon[31173]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:07:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:00 smithi092 ceph-mon[31173]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:07:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:00 smithi092 ceph-mon[31173]: from='osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:07:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:00 smithi092 ceph-mon[31173]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:07:01.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:01.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:00 smithi092 ceph-mon[31173]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:07:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:07:01.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 -- 172.21.15.153:0/1284449560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288101cd0 msgr2=0x7fa288102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:01.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 --2- 172.21.15.153:0/1284449560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288101cd0 0x7fa288102150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa270009900 tx=0x7fa27002a410 comp rx=0 tx=0).stop 2024-09-11T22:07:01.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 -- 172.21.15.153:0/1284449560 shutdown_connections 2024-09-11T22:07:01.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 --2- 172.21.15.153:0/1284449560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa288102690 0x7fa288108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:01.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 --2- 172.21.15.153:0/1284449560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288101cd0 0x7fa288102150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 --2- 172.21.15.153:0/1284449560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa288100ad0 0x7fa288100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 -- 172.21.15.153:0/1284449560 >> 172.21.15.153:0/1284449560 conn(0x7fa2880fc280 msgr2=0x7fa2880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 -- 172.21.15.153:0/1284449560 shutdown_connections 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 -- 172.21.15.153:0/1284449560 wait complete. 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.639+0000 7fa28cb44640 1 Processor -- start 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 -- start start 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288100ad0 0x7fa288197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa288101cd0 0x7fa2881982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa288102690 0x7fa28819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d730 con 0x7fa288102690 2024-09-11T22:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d5b0 con 0x7fa288101cd0 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d8b0 con 0x7fa288100ad0 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288100ad0 0x7fa288197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288100ad0 0x7fa288197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44546/0 (socket says 172.21.15.153:44546) 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 -- 172.21.15.153:0/248803203 learned_addr learned my addr 172.21.15.153:0/248803203 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa27dd9b640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa288101cd0 0x7fa2881982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa286d9d640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa288102690 0x7fa28819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 -- 172.21.15.153:0/248803203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa288101cd0 msgr2=0x7fa2881982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa288101cd0 0x7fa2881982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 -- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa288102690 msgr2=0x7fa28819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa288102690 0x7fa28819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 -- 172.21.15.153:0/248803203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28819f9e0 con 0x7fa288100ad0 2024-09-11T22:07:01.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa286d9d640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa288102690 0x7fa28819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28659c640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288100ad0 0x7fa288197db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa27800d770 tx=0x7fa27800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa27ffff640 1 -- 172.21.15.153:0/248803203 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278017070 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28819fcd0 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.640+0000 7fa27ffff640 1 -- 172.21.15.153:0/248803203 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa278009c30 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.641+0000 7fa27ffff640 1 -- 172.21.15.153:0/248803203 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278003d00 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.641+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2881a0190 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.641+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa288107f20 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.643+0000 7fa27ffff640 1 -- 172.21.15.153:0/248803203 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa278003ea0 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.643+0000 7fa27ffff640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2740a2b20 0x7fa2740a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.643+0000 7fa27ffff640 1 -- 172.21.15.153:0/248803203 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fa2780cd000 con 0x7fa288100ad0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.643+0000 7fa27dd9b640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2740a2b20 0x7fa2740a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:01.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.644+0000 7fa27dd9b640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2740a2b20 0x7fa2740a4fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa270006010 tx=0x7fa270000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:01.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.648+0000 7fa27ffff640 1 -- 172.21.15.153:0/248803203 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa27809b8d0 con 0x7fa288100ad0 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:01.827 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:01.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:01.826+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa288062630 con 0x7fa2740a2b20 2024-09-11T22:07:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:07:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:07:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:02.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:02.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:02.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:02.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:07:02.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:07:02.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi110 2024-09-11T22:07:02.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-11T22:07:02.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:07:02.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] boot 2024-09-11T22:07:02.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:07:02.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:02.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:02.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:02.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:07:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:07:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi110 2024-09-11T22:07:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-11T22:07:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:07:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] boot 2024-09-11T22:07:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:07:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:07:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:07:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi110 2024-09-11T22:07:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-11T22:07:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:07:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: osd.1 [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] boot 2024-09-11T22:07:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:07:03.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:07:03.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:03.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:03.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:03.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:03 smithi153 ceph-mon[35045]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:03.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:03 smithi153 ceph-mon[35045]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:07:04.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:03 smithi092 ceph-mon[31173]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:04.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:03 smithi092 ceph-mon[31173]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:07:04.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:03 smithi110 ceph-mon[34830]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:04.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:03 smithi110 ceph-mon[34830]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:07:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:04 smithi092 ceph-mon[31173]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:04 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/2857410674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]: dispatch 2024-09-11T22:07:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:04 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]: dispatch 2024-09-11T22:07:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:04 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]': finished 2024-09-11T22:07:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:04 smithi092 ceph-mon[31173]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:07:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:04 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/2484612707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:05.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:04 smithi153 ceph-mon[35045]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:05.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:04 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/2857410674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]: dispatch 2024-09-11T22:07:05.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:04 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]: dispatch 2024-09-11T22:07:05.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:04 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]': finished 2024-09-11T22:07:05.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:04 smithi153 ceph-mon[35045]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:07:05.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:05.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:04 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/2484612707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:04 smithi110 ceph-mon[34830]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:04 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/2857410674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]: dispatch 2024-09-11T22:07:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:04 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]: dispatch 2024-09-11T22:07:05.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:04 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1be9b69c-ef19-405e-9f42-0964e338d4c9"}]': finished 2024-09-11T22:07:05.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:04 smithi110 ceph-mon[34830]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:07:05.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:05.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:04 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/2484612707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:07.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:06 smithi092 ceph-mon[31173]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:07.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:06 smithi153 ceph-mon[35045]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:06 smithi110 ceph-mon[34830]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:09.008 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:08 smithi153 ceph-mon[35045]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:08 smithi092 ceph-mon[31173]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:08 smithi110 ceph-mon[34830]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:11.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:10 smithi092 ceph-mon[31173]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:11.026 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:10 smithi153 ceph-mon[35045]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:10 smithi110 ceph-mon[34830]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:13.011 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:12 smithi153 ceph-mon[35045]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:13.011 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:07:13.011 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:12 smithi092 ceph-mon[31173]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:07:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:12 smithi110 ceph-mon[34830]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:07:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:14.001 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:13 smithi153 ceph-mon[35045]: Deploying daemon osd.2 on smithi153 2024-09-11T22:07:14.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:13 smithi092 ceph-mon[31173]: Deploying daemon osd.2 on smithi153 2024-09-11T22:07:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:13 smithi110 ceph-mon[34830]: Deploying daemon osd.2 on smithi153 2024-09-11T22:07:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:14 smithi092 ceph-mon[31173]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:14 smithi153 ceph-mon[35045]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:14 smithi110 ceph-mon[34830]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:16 smithi092 ceph-mon[31173]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:16 smithi153 ceph-mon[35045]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:16 smithi110 ceph-mon[34830]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:18 smithi092 ceph-mon[31173]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:19.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:18 smithi153 ceph-mon[35045]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:18 smithi110 ceph-mon[34830]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:19 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:19 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:20.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:19 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:20.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:19 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:20.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:19 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:20.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:19 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:19 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:19 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:19 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:20 smithi092 ceph-mon[31173]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:21.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:20 smithi153 ceph-mon[35045]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:20 smithi110 ceph-mon[34830]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:22.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:22.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:22.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:22 smithi092 ceph-mon[31173]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:22 smithi153 ceph-mon[35045]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:22 smithi110 ceph-mon[34830]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:23.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.040+0000 7fa27ffff640 1 -- 172.21.15.153:0/248803203 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa288062630 con 0x7fa2740a2b20 2024-09-11T22:07:23.041 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 2 on host 'smithi153' 2024-09-11T22:07:23.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.042+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2740a2b20 msgr2=0x7fa2740a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:23.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.042+0000 7fa28cb44640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2740a2b20 0x7fa2740a4fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa270006010 tx=0x7fa270000bc0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.042+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288100ad0 msgr2=0x7fa288197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:23.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.042+0000 7fa28cb44640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288100ad0 0x7fa288197db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa27800d770 tx=0x7fa27800dc30 comp rx=0 tx=0).stop 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 shutdown_connections 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2740a2b20 0x7fa2740a4fe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa288102690 0x7fa28819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa288101cd0 0x7fa2881982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 --2- 172.21.15.153:0/248803203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa288100ad0 0x7fa288197db0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 >> 172.21.15.153:0/248803203 conn(0x7fa2880fc280 msgr2=0x7fa2881050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 shutdown_connections 2024-09-11T22:07:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:07:23.043+0000 7fa28cb44640 1 -- 172.21.15.153:0/248803203 wait complete. 2024-09-11T22:07:23.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:23.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:23.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:23.618 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:23.619 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:23.619 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:23.687 DEBUG:teuthology.orchestra.run.smithi153:osd.2> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T22:07:23.690 INFO:tasks.cephadm:Deploying osd.3 on smithi092 with /dev/vg_nvme/lv_3... 2024-09-11T22:07:23.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:07:23.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:23.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:23.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:23.928 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:24.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:24 smithi092 ceph-mon[31173]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:24.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:24 smithi153 ceph-mon[35045]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:24.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:24 smithi110 ceph-mon[34830]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:25.090 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:24 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2[39997]: 2024-09-11T22:07:24.869+0000 7fa6a5177640 -1 osd.2 0 log_to_monitors true 2024-09-11T22:07:25.358 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:25 smithi153 ceph-mon[35045]: from='osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:07:25.358 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:25 smithi153 ceph-mon[35045]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:07:25.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:25 smithi092 ceph-mon[31173]: from='osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:07:25.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:25 smithi092 ceph-mon[31173]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:07:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:25 smithi110 ceph-mon[34830]: from='osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:07:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:25 smithi110 ceph-mon[34830]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:07:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:07:26.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:26 smithi092 ceph-mon[31173]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:07:26.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:26 smithi092 ceph-mon[31173]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:07:26.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:26 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:26.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:26 smithi092 ceph-mon[31173]: from='osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:07:26.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:26 smithi092 ceph-mon[31173]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:07:26.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:26 smithi092 ceph-mon[31173]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:26.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-09-11T22:07:26.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-mon[35045]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:07:26.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-mon[35045]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:07:26.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:26.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-mon[35045]: from='osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:07:26.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-mon[35045]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:07:26.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-mon[35045]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:26 smithi110 ceph-mon[34830]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:07:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:26 smithi110 ceph-mon[34830]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:07:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:26 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:26 smithi110 ceph-mon[34830]: from='osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:07:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:26 smithi110 ceph-mon[34830]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:07:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:26 smithi110 ceph-mon[34830]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:27.039 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2[39997]: 2024-09-11T22:07:26.717+0000 7fa6a190e640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T22:07:27.039 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2[39997]: 2024-09-11T22:07:26.720+0000 7fa698f1a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 -- 172.21.15.92:0/628817094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 msgr2=0x7f180c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 --2- 172.21.15.92:0/628817094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c10a050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1804009ca0 tx=0x7f180402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 -- 172.21.15.92:0/628817094 shutdown_connections 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 --2- 172.21.15.92:0/628817094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c10a050 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 --2- 172.21.15.92:0/628817094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f180c102e00 0x7f180c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 --2- 172.21.15.92:0/628817094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f180c101c00 0x7f180c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 -- 172.21.15.92:0/628817094 >> 172.21.15.92:0/628817094 conn(0x7f180c0fd390 msgr2=0x7f180c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 -- 172.21.15.92:0/628817094 shutdown_connections 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.051+0000 7f18124f6640 1 -- 172.21.15.92:0/628817094 wait complete. 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 Processor -- start 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 -- start start 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f180c101c00 0x7f180c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f180c102e00 0x7f180c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10d7b0 con 0x7f180c1037c0 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10d630 con 0x7f180c102e00 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f18124f6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10d930 con 0x7f180c101c00 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f1810a93640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f1810a93640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40414/0 (socket says 172.21.15.92:40414) 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f1810a93640 1 -- 172.21.15.92:0/3011639352 learned_addr learned my addr 172.21.15.92:0/3011639352 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f180bfff640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f180c101c00 0x7f180c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.052+0000 7f180b7fe640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f180c102e00 0x7f180c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f1810a93640 1 -- 172.21.15.92:0/3011639352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f180c101c00 msgr2=0x7f180c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f1810a93640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f180c101c00 0x7f180c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f1810a93640 1 -- 172.21.15.92:0/3011639352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f180c102e00 msgr2=0x7f180c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f1810a93640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f180c102e00 0x7f180c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f1810a93640 1 -- 172.21.15.92:0/3011639352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c19fa30 con 0x7f180c1037c0 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f180b7fe640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f180c102e00 0x7f180c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f1810a93640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c19f3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1804009dc0 tx=0x7f1804003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804004040 con 0x7f180c1037c0 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1804004690 con 0x7f180c1037c0 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c19fcc0 con 0x7f180c1037c0 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180402f4c0 con 0x7f180c1037c0 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.053+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1a01b0 con 0x7f180c1037c0 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.054+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180c109000 con 0x7f180c1037c0 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.055+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f18040305c0 con 0x7f180c1037c0 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.056+0000 7f18097fa640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f17f40a2b20 0x7f17f40a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.056+0000 7f180bfff640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f17f40a2b20 0x7f17f40a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.056+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f18040ef070 con 0x7f180c1037c0 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.056+0000 7f180bfff640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f17f40a2b20 0x7f17f40a4fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f180c102c60 tx=0x7f17fc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.064+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18040bd720 con 0x7f180c1037c0 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:27.240+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f180c062630 con 0x7f17f40a2b20 2024-09-11T22:07:27.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi153 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='osd.2 ' entity='osd.2' 2024-09-11T22:07:27.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:27.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:27.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:27 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:27.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:07:27.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:07:27.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:27.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:27.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi153 2024-09-11T22:07:27.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='osd.2 ' entity='osd.2' 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:27.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:27 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:27.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:07:27.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:07:27.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:27.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:27.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi153 2024-09-11T22:07:27.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='osd.2 ' entity='osd.2' 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:27.790 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:27 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:28.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:07:28.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:07:28.592 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:28.592 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:28.592 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:28.592 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] boot 2024-09-11T22:07:28.592 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:07:28.592 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] boot 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:07:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: osd.2 [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] boot 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:07:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:07:30.154 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:29 smithi092 ceph-mon[31173]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:07:30.154 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:29 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3358878711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0"}]: dispatch 2024-09-11T22:07:30.154 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:29 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3358878711' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0"}]': finished 2024-09-11T22:07:30.154 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:29 smithi092 ceph-mon[31173]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T22:07:30.154 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:29 smithi110 ceph-mon[34830]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:07:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:29 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3358878711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0"}]: dispatch 2024-09-11T22:07:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:29 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3358878711' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0"}]': finished 2024-09-11T22:07:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:29 smithi110 ceph-mon[34830]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T22:07:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:30.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:29 smithi153 ceph-mon[35045]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:07:30.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:29 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3358878711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0"}]: dispatch 2024-09-11T22:07:30.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:29 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3358878711' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0"}]': finished 2024-09-11T22:07:30.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:29 smithi153 ceph-mon[35045]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T22:07:30.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:30 smithi110 ceph-mon[34830]: pgmap v77: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:30 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:30 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2917943745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:30 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:07:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:30 smithi110 ceph-mon[34830]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:07:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:30 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:31.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:30 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:31.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:30 smithi092 ceph-mon[31173]: pgmap v77: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:31.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:30 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:31.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:30 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2917943745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:31.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:30 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:07:31.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:30 smithi092 ceph-mon[31173]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:07:31.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:30 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:31.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:30 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:30 smithi153 ceph-mon[35045]: pgmap v77: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:30 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:30 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2917943745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:30 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:07:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:30 smithi153 ceph-mon[35045]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:07:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:30 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:30 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:32.011 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:31 smithi092 sudo[50504]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:07:32.011 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:31 smithi092 sudo[50504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:32.011 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:31 smithi092 sudo[50504]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:32.011 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:31 smithi092 sudo[50504]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:32.268 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:32 smithi092 sudo[50509]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:07:32.268 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:32 smithi092 sudo[50509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:32.268 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:32 smithi092 sudo[50509]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:32.268 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:07:32 smithi092 sudo[50509]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:32.429+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f18040b9690 con 0x7f180c1037c0 2024-09-11T22:07:32.534 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:07:32.534 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:32 smithi110 ceph-mon[34830]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:07:32.534 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:32.535 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:32 smithi110 ceph-mon[34830]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:32.535 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:32.535 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43114]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:07:32.535 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43114]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:32.535 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43114]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:32.535 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43114]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:32.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:07:32.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:32 smithi092 ceph-mon[31173]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:07:32.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:32.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:32 smithi092 ceph-mon[31173]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:32.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:07:32.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:32 smithi153 ceph-mon[35045]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:07:32.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:32.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:32 smithi153 ceph-mon[35045]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:32.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:32.967 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43117]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:07:32.967 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:32.967 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43117]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:32.967 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:07:32 smithi110 sudo[43117]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:33.056 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:32 smithi153 sudo[44018]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:07:33.056 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:32 smithi153 sudo[44018]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:33.056 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:32 smithi153 sudo[44018]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:33.056 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:32 smithi153 sudo[44018]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:33.538 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:07:33.539 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:33.539 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44021]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:33.539 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44021]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:33.581 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[50883]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:07:33.581 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[50883]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:33.581 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[50883]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:33.581 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[50883]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:33.846 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 ceph-mon[35045]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:07:33.846 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 ceph-mon[35045]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:07:33.846 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:33.847 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:33.847 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:33.847 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 ceph-mon[35045]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:33.847 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44024]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:07:33.847 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44024]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:33.847 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44024]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:33.847 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:33 smithi153 sudo[44024]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:33 smithi110 ceph-mon[34830]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:07:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:33 smithi110 ceph-mon[34830]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:07:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:33.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:33 smithi110 ceph-mon[34830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:34.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[51070]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:07:34.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[51070]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:34.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[51070]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:34.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 sudo[51070]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:34.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 ceph-mon[31173]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:07:34.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 ceph-mon[31173]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:07:34.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:34.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:34.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:34.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:33 smithi092 ceph-mon[31173]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:34.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 sudo[44027]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:07:34.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 sudo[44027]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:34.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 sudo[44027]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:34.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 sudo[44027]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:34.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43120]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:07:34.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43120]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:34.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43120]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:34.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43120]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:34.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:34.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:34.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:34.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:34 smithi092 ceph-mon[31173]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43123]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:07:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43123]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:07:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43123]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:07:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 sudo[43123]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:07:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:34.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:34.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:34 smithi110 ceph-mon[34830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:35.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:35.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:07:35.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:07:35.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:07:35.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:07:35.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:34 smithi153 ceph-mon[35045]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:07:36.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:36 smithi092 ceph-mon[31173]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:36.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:36 smithi153 ceph-mon[35045]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:36.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:36 smithi110 ceph-mon[34830]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:38.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:38 smithi110 ceph-mon[34830]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:38.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:07:38.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:38.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:38 smithi110 ceph-mon[34830]: Deploying daemon osd.3 on smithi092 2024-09-11T22:07:39.009 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:38 smithi092 ceph-mon[31173]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:39.009 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:07:39.009 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:39.009 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:38 smithi092 ceph-mon[31173]: Deploying daemon osd.3 on smithi092 2024-09-11T22:07:39.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:38 smithi153 ceph-mon[35045]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:39.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:07:39.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:39.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:38 smithi153 ceph-mon[35045]: Deploying daemon osd.3 on smithi092 2024-09-11T22:07:40.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:40 smithi092 ceph-mon[31173]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T22:07:40.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:40 smithi110 ceph-mon[34830]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T22:07:41.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:40 smithi153 ceph-mon[35045]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T22:07:42.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:42 smithi092 ceph-mon[31173]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T22:07:42.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:42 smithi110 ceph-mon[34830]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T22:07:43.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:42 smithi153 ceph-mon[35045]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T22:07:44.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:44 smithi092 ceph-mon[31173]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T22:07:44.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:44.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:44.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:44.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:44 smithi110 ceph-mon[34830]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T22:07:44.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:44.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:44.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:44 smithi153 ceph-mon[35045]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T22:07:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:46.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:46 smithi110 ceph-mon[34830]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:46.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:46 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:46.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:46 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:46.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:46 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:46.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:46 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:46.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:46 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:46 smithi092 ceph-mon[31173]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:46 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:46 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:46 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:47.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:46 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:47.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:46 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:46 smithi153 ceph-mon[35045]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:47.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:46 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:46 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:46 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:47.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:46 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:47.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:46 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.405+0000 7f18097fa640 1 -- 172.21.15.92:0/3011639352 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f180c062630 con 0x7f17f40a2b20 2024-09-11T22:07:47.406 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 3 on host 'smithi092' 2024-09-11T22:07:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f17f40a2b20 msgr2=0x7f17f40a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f17f40a2b20 0x7f17f40a4fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f180c102c60 tx=0x7f17fc009110 comp rx=0 tx=0).stop 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 msgr2=0x7f180c19f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c19f3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1804009dc0 tx=0x7f1804003ba0 comp rx=0 tx=0).stop 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 shutdown_connections 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f17f40a2b20 0x7f17f40a4fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1037c0 0x7f180c19f3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f180c102e00 0x7f180c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 --2- 172.21.15.92:0/3011639352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f180c101c00 0x7f180c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 >> 172.21.15.92:0/3011639352 conn(0x7f180c0fd390 msgr2=0x7f180c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.407+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 shutdown_connections 2024-09-11T22:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:07:47.408+0000 7f18124f6640 1 -- 172.21.15.92:0/3011639352 wait complete. 2024-09-11T22:07:47.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:47.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:47.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:48.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:48.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:48.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:48.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:48.062 DEBUG:teuthology.orchestra.run.smithi092:osd.3> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T22:07:48.065 INFO:tasks.cephadm:Deploying osd.4 on smithi110 with /dev/vg_nvme/lv_3... 2024-09-11T22:07:48.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:07:48.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:48 smithi110 ceph-mon[34830]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:49.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:48 smithi092 ceph-mon[31173]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:48 smithi153 ceph-mon[35045]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:49.466 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:07:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:07:49.317+0000 7fb1c9baa640 -1 osd.3 0 log_to_monitors true 2024-09-11T22:07:49.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:49 smithi092 ceph-mon[31173]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:07:49.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:49 smithi110 ceph-mon[34830]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:07:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:49 smithi153 ceph-mon[35045]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:07:50.937 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:07:50.973 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-09-11T22:07:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:50 smithi110 ceph-mon[34830]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:50 smithi110 ceph-mon[34830]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:07:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:50 smithi110 ceph-mon[34830]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T22:07:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:50 smithi110 ceph-mon[34830]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:07:51.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:51.232 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:07:51.268 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:07:51.097+0000 7fb1c6341640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T22:07:51.268 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:07:51.102+0000 7fb1bd94d640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:07:51.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:50 smithi092 ceph-mon[31173]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:51.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:50 smithi092 ceph-mon[31173]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:07:51.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:50 smithi092 ceph-mon[31173]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T22:07:51.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:50 smithi092 ceph-mon[31173]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:07:51.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:51.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:50 smithi153 ceph-mon[35045]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T22:07:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:50 smithi153 ceph-mon[35045]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:07:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:50 smithi153 ceph-mon[35045]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T22:07:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:50 smithi153 ceph-mon[35045]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:07:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 -- 172.21.15.110:0/321482966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c1037c0 msgr2=0x7fd25c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 --2- 172.21.15.110:0/321482966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c1037c0 0x7fd25c10a050 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd254009ca0 tx=0x7fd25402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 -- 172.21.15.110:0/321482966 shutdown_connections 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 --2- 172.21.15.110:0/321482966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c1037c0 0x7fd25c10a050 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 --2- 172.21.15.110:0/321482966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd25c102e00 0x7fd25c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 --2- 172.21.15.110:0/321482966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd25c101c00 0x7fd25c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 -- 172.21.15.110:0/321482966 >> 172.21.15.110:0/321482966 conn(0x7fd25c0fd390 msgr2=0x7fd25c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 -- 172.21.15.110:0/321482966 shutdown_connections 2024-09-11T22:07:51.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.687+0000 7fd26449c640 1 -- 172.21.15.110:0/321482966 wait complete. 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 Processor -- start 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 -- start start 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd25c101c00 0x7fd25c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c102e00 0x7fd25c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd25c1037c0 0x7fd25c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c10d7b0 con 0x7fd25c1037c0 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c10d630 con 0x7fd25c102e00 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd26449c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c10d930 con 0x7fd25c101c00 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c102e00 0x7fd25c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c102e00 0x7fd25c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47530/0 (socket says 172.21.15.110:47530) 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 -- 172.21.15.110:0/2848891122 learned_addr learned my addr 172.21.15.110:0/2848891122 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T22:07:51.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd262a39640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd25c1037c0 0x7fd25c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 -- 172.21.15.110:0/2848891122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd25c101c00 msgr2=0x7fd25c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd25c101c00 0x7fd25c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 -- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd25c1037c0 msgr2=0x7fd25c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd25c1037c0 0x7fd25c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd261a37640 1 -- 172.21.15.110:0/2848891122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c19fa60 con 0x7fd25c102e00 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.688+0000 7fd262a39640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd25c1037c0 0x7fd25c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.689+0000 7fd261a37640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c102e00 0x7fd25c198370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00d950 tx=0x7fd24c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.689+0000 7fd2537fe640 1 -- 172.21.15.110:0/2848891122 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c017070 con 0x7fd25c102e00 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.689+0000 7fd2537fe640 1 -- 172.21.15.110:0/2848891122 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd24c009c30 con 0x7fd25c102e00 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.689+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c19fd50 con 0x7fd25c102e00 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.689+0000 7fd2537fe640 1 -- 172.21.15.110:0/2848891122 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00f620 con 0x7fd25c102e00 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.689+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c1a0230 con 0x7fd25c102e00 2024-09-11T22:07:51.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.690+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c109050 con 0x7fd25c102e00 2024-09-11T22:07:51.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.691+0000 7fd2537fe640 1 -- 172.21.15.110:0/2848891122 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd24c0109a0 con 0x7fd25c102e00 2024-09-11T22:07:51.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.691+0000 7fd2537fe640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd2480a2bb0 0x7fd2480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:51.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.692+0000 7fd2537fe640 1 -- 172.21.15.110:0/2848891122 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3422+0+0 (secure 0 0 0) 0x7fd24c0cd9d0 con 0x7fd25c102e00 2024-09-11T22:07:51.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.692+0000 7fd262238640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd2480a2bb0 0x7fd2480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:51.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.692+0000 7fd262238640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd2480a2bb0 0x7fd2480a5070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd25c102c60 tx=0x7fd258006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:51.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.696+0000 7fd2537fe640 1 -- 172.21.15.110:0/2848891122 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd24c009de0 con 0x7fd25c102e00 2024-09-11T22:07:51.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:07:51.870+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd25c062630 con 0x7fd2480a2bb0 2024-09-11T22:07:52.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:07:52.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T22:07:52.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi092 2024-09-11T22:07:52.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: osdmap e26: 4 total, 3 up, 4 in 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:52.219 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:52.219 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi092 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-11T22:07:52.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: osdmap e26: 4 total, 3 up, 4 in 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:52.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi092 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-11T22:07:52.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: osdmap e26: 4 total, 3 up, 4 in 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439]' entity='osd.3' 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:07:52.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:53.211 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:53 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:07:53.211 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:53 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:07:53.211 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:53 smithi110 ceph-mon[34830]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:53.211 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:53 smithi110 ceph-mon[34830]: osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] boot 2024-09-11T22:07:53.211 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:53 smithi110 ceph-mon[34830]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:07:53.211 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:53 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:53.437 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:53 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:07:53.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:53 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:07:53.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:53 smithi092 ceph-mon[31173]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:53.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:53 smithi092 ceph-mon[31173]: osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] boot 2024-09-11T22:07:53.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:53 smithi092 ceph-mon[31173]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:07:53.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:53 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:53.459 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:53 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:07:53.459 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:53 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:07:53.459 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:53 smithi153 ceph-mon[35045]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:53.459 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:53 smithi153 ceph-mon[35045]: osd.3 [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] boot 2024-09-11T22:07:53.460 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:53 smithi153 ceph-mon[35045]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:07:53.460 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:53 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:07:54.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:54 smithi110 ceph-mon[34830]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T22:07:54.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:54 smithi110 ceph-mon[34830]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:54.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:54 smithi110 ceph-mon[34830]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T22:07:54.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:54 smithi092 ceph-mon[31173]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T22:07:54.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:54 smithi092 ceph-mon[31173]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:54.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:54 smithi092 ceph-mon[31173]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T22:07:54.425 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:54 smithi153 ceph-mon[35045]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T22:07:54.425 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:54 smithi153 ceph-mon[35045]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:54.425 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:54 smithi153 ceph-mon[35045]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T22:07:55.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:55 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/1371786025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]: dispatch 2024-09-11T22:07:55.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:55 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]: dispatch 2024-09-11T22:07:55.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:55 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]': finished 2024-09-11T22:07:55.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:55 smithi110 ceph-mon[34830]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:07:55.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:55.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:55 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/383407053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:55.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:55 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/1371786025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]: dispatch 2024-09-11T22:07:55.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:55 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]: dispatch 2024-09-11T22:07:55.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:55 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]': finished 2024-09-11T22:07:55.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:55 smithi092 ceph-mon[31173]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:07:55.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:55.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:55 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/383407053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:55.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:55 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/1371786025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]: dispatch 2024-09-11T22:07:55.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:55 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]: dispatch 2024-09-11T22:07:55.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:55 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9039a81-37d0-4a40-8050-394638ea88a5"}]': finished 2024-09-11T22:07:55.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:55 smithi153 ceph-mon[35045]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:07:55.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:55.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:55 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/383407053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:07:56.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:56 smithi110 ceph-mon[34830]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T22:07:56.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:56 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:56.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:56 smithi110 ceph-mon[34830]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:56.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:56 smithi092 ceph-mon[31173]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T22:07:56.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:56 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:56.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:56 smithi092 ceph-mon[31173]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:56.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:56 smithi153 ceph-mon[35045]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T22:07:56.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:56 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:56.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:56 smithi153 ceph-mon[35045]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:57.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:57 smithi110 ceph-mon[34830]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T22:07:57.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:57.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:57 smithi092 ceph-mon[31173]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T22:07:57.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:57.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:57 smithi153 ceph-mon[35045]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T22:07:57.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:07:58.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:07:58 smithi110 ceph-mon[34830]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:58.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:07:58 smithi092 ceph-mon[31173]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:07:58.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:07:58 smithi153 ceph-mon[35045]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:00.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:00 smithi110 ceph-mon[34830]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-11T22:08:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:00 smithi092 ceph-mon[31173]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-11T22:08:01.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:00 smithi153 ceph-mon[35045]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-11T22:08:02.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:02 smithi110 ceph-mon[34830]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T22:08:02.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:08:02.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:02 smithi092 ceph-mon[31173]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T22:08:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:08:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:03.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:02 smithi153 ceph-mon[35045]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T22:08:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:08:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:03.866 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:03 smithi110 ceph-mon[34830]: Deploying daemon osd.4 on smithi110 2024-09-11T22:08:04.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:03 smithi092 ceph-mon[31173]: Deploying daemon osd.4 on smithi110 2024-09-11T22:08:04.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:03 smithi153 ceph-mon[35045]: Deploying daemon osd.4 on smithi110 2024-09-11T22:08:04.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:04 smithi092 ceph-mon[31173]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T22:08:04.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:04 smithi153 ceph-mon[35045]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T22:08:04.805 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:04 smithi110 ceph-mon[34830]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T22:08:06.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:06 smithi110 ceph-mon[34830]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:08:07.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:06 smithi092 ceph-mon[31173]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:08:07.054 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:06 smithi153 ceph-mon[35045]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:08:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:08 smithi110 ceph-mon[34830]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:08:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:08.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:08 smithi092 ceph-mon[31173]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:08:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:08 smithi153 ceph-mon[35045]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:08:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:10.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:10 smithi110 ceph-mon[34830]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:08:11.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:10 smithi092 ceph-mon[31173]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:08:11.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:10 smithi153 ceph-mon[35045]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:08:12.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.234+0000 7fd2537fe640 1 -- 172.21.15.110:0/2848891122 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd25c062630 con 0x7fd2480a2bb0 2024-09-11T22:08:12.235 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 4 on host 'smithi110' 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd2480a2bb0 msgr2=0x7fd2480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd2480a2bb0 0x7fd2480a5070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd25c102c60 tx=0x7fd258006c90 comp rx=0 tx=0).stop 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c102e00 msgr2=0x7fd25c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c102e00 0x7fd25c198370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00d950 tx=0x7fd24c00de10 comp rx=0 tx=0).stop 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 shutdown_connections 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd2480a2bb0 0x7fd2480a5070 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd25c1037c0 0x7fd25c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd25c102e00 0x7fd25c198370 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:12.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 --2- 172.21.15.110:0/2848891122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd25c101c00 0x7fd25c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:12.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 >> 172.21.15.110:0/2848891122 conn(0x7fd25c0fd390 msgr2=0x7fd25c106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:12.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 shutdown_connections 2024-09-11T22:08:12.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:08:12.236+0000 7fd26449c640 1 -- 172.21.15.110:0/2848891122 wait complete. 2024-09-11T22:08:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:12.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:12.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:12.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:12.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:12.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:12.903 DEBUG:teuthology.orchestra.run.smithi110:osd.4> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T22:08:12.906 INFO:tasks.cephadm:Deploying osd.5 on smithi153 with /dev/vg_nvme/lv_3... 2024-09-11T22:08:12.906 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:08:13.160 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:08:14.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:14 smithi153 ceph-mon[35045]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:14.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:14 smithi153 ceph-mon[35045]: from='osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:08:14.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:14 smithi153 ceph-mon[35045]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:08:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:14 smithi110 ceph-mon[34830]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:14 smithi110 ceph-mon[34830]: from='osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:08:14.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:14 smithi110 ceph-mon[34830]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:08:14.968 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:08:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:08:14.658+0000 7fb66ea04640 -1 osd.4 0 log_to_monitors true 2024-09-11T22:08:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:14 smithi092 ceph-mon[31173]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:14 smithi092 ceph-mon[31173]: from='osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:08:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:14 smithi092 ceph-mon[31173]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:08:15.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:15 smithi153 ceph-mon[35045]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:08:15.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:15 smithi153 ceph-mon[35045]: from='osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:08:15.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:15 smithi153 ceph-mon[35045]: osdmap e33: 5 total, 4 up, 5 in 2024-09-11T22:08:15.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:15.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:15 smithi153 ceph-mon[35045]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:08:15.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:15 smithi153 ceph-mon[35045]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:08:15.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:08:15.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2024-09-11T22:08:15.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-mon[34830]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:08:15.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-mon[34830]: from='osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:08:15.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-mon[34830]: osdmap e33: 5 total, 4 up, 5 in 2024-09-11T22:08:15.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:15.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-mon[34830]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:08:15.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-mon[34830]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:08:15.967 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:08:15.695+0000 7fb66a987640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T22:08:15.967 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:08:15 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:08:15.701+0000 7fb6667af640 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:08:16.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:15 smithi092 ceph-mon[31173]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:08:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:15 smithi092 ceph-mon[31173]: from='osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:08:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:15 smithi092 ceph-mon[31173]: osdmap e33: 5 total, 4 up, 5 in 2024-09-11T22:08:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:15 smithi092 ceph-mon[31173]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:08:16.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:15 smithi092 ceph-mon[31173]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:08:16.212 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:08:16.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 -- 172.21.15.153:0/1881716492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900102e00 msgr2=0x7fe900103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:16.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1881716492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900102e00 0x7fe900103280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009900 tx=0x7fe8f402a410 comp rx=0 tx=0).stop 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 -- 172.21.15.153:0/1881716492 shutdown_connections 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1881716492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9001037c0 0x7fe90010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1881716492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900102e00 0x7fe900103280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1881716492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe900101c00 0x7fe900102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 -- 172.21.15.153:0/1881716492 >> 172.21.15.153:0/1881716492 conn(0x7fe9000fd390 msgr2=0x7fe9000ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 -- 172.21.15.153:0/1881716492 shutdown_connections 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 -- 172.21.15.153:0/1881716492 wait complete. 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.697+0000 7fe905b5f640 1 Processor -- start 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe905b5f640 1 -- start start 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe905b5f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900101c00 0x7fe900197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:16.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe905b5f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe900102e00 0x7fe900198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe905b5f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9001037c0 0x7fe90019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe905b5f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010d780 con 0x7fe900102e00 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe905b5f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010d600 con 0x7fe9001037c0 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe905b5f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010d900 con 0x7fe900101c00 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900101c00 0x7fe900197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900101c00 0x7fe900197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39230/0 (socket says 172.21.15.153:39230) 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 -- 172.21.15.153:0/1372012606 learned_addr learned my addr 172.21.15.153:0/1372012606 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8feffd640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe900102e00 0x7fe900198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8fffff640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9001037c0 0x7fe90019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 -- 172.21.15.153:0/1372012606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9001037c0 msgr2=0x7fe90019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9001037c0 0x7fe90019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:16.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 -- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe900102e00 msgr2=0x7fe900198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe900102e00 0x7fe900198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8ff7fe640 1 -- 172.21.15.153:0/1372012606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90019fa40 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.698+0000 7fe8feffd640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe900102e00 0x7fe900198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.699+0000 7fe8ff7fe640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900101c00 0x7fe900197e10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000eaa0 tx=0x7fe8f000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.699+0000 7fe8fcff9640 1 -- 172.21.15.153:0/1372012606 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0016070 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.699+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90019fd30 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.699+0000 7fe8fcff9640 1 -- 172.21.15.153:0/1372012606 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8f0003e20 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.699+0000 7fe8fcff9640 1 -- 172.21.15.153:0/1372012606 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00116d0 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.699+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9001a0190 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.700+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe900108ff0 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.701+0000 7fe8fcff9640 1 -- 172.21.15.153:0/1372012606 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8f0019070 con 0x7fe900101c00 2024-09-11T22:08:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.702+0000 7fe8fcff9640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe8e80a2bb0 0x7fe8e80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:16.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.702+0000 7fe8fcff9640 1 -- 172.21.15.153:0/1372012606 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 3821+0+0 (secure 0 0 0) 0x7fe8f00ce000 con 0x7fe900101c00 2024-09-11T22:08:16.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.702+0000 7fe8feffd640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe8e80a2bb0 0x7fe8e80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:16.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.702+0000 7fe8feffd640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe8e80a2bb0 0x7fe8e80a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000c00 tx=0x7fe8f40314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:16.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.711+0000 7fe8fcff9640 1 -- 172.21.15.153:0/1372012606 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8f009c0f0 con 0x7fe900101c00 2024-09-11T22:08:16.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:16.889+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe900062630 con 0x7fe8e80a2bb0 2024-09-11T22:08:16.889 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: osdmap e34: 5 total, 4 up, 5 in 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi110 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.890 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: osdmap e34: 5 total, 4 up, 5 in 2024-09-11T22:08:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi110 2024-09-11T22:08:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:16.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: osdmap e34: 5 total, 4 up, 5 in 2024-09-11T22:08:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi110 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] boot 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: osdmap e35: 5 total, 5 up, 5 in 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:08:17.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:17 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:08:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:08:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] boot 2024-09-11T22:08:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: osdmap e35: 5 total, 5 up, 5 in 2024-09-11T22:08:18.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:18.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:08:18.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:08:18.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:17 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:08:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:08:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: osd.4 [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] boot 2024-09-11T22:08:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: osdmap e35: 5 total, 5 up, 5 in 2024-09-11T22:08:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:08:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:08:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:08:18.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:17 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:18.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:18 smithi153 ceph-mon[35045]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:08:18.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:18 smithi153 ceph-mon[35045]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:18.945 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:18 smithi153 ceph-mon[35045]: osdmap e36: 5 total, 5 up, 5 in 2024-09-11T22:08:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:18 smithi092 ceph-mon[31173]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:08:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:18 smithi092 ceph-mon[31173]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:18 smithi092 ceph-mon[31173]: osdmap e36: 5 total, 5 up, 5 in 2024-09-11T22:08:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:18 smithi110 ceph-mon[34830]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:08:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:18 smithi110 ceph-mon[34830]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:08:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:18 smithi110 ceph-mon[34830]: osdmap e36: 5 total, 5 up, 5 in 2024-09-11T22:08:19.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:19 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/3647315880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]: dispatch 2024-09-11T22:08:19.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:19 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]: dispatch 2024-09-11T22:08:19.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:19 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]': finished 2024-09-11T22:08:19.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:19 smithi153 ceph-mon[35045]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T22:08:19.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:19 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:19 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/3647315880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]: dispatch 2024-09-11T22:08:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:19 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]: dispatch 2024-09-11T22:08:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:19 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]': finished 2024-09-11T22:08:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:19 smithi092 ceph-mon[31173]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T22:08:20.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:19 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:19 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/3647315880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]: dispatch 2024-09-11T22:08:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:19 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]: dispatch 2024-09-11T22:08:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:19 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0472991-0835-438f-9579-8e10646de923"}]': finished 2024-09-11T22:08:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:19 smithi110 ceph-mon[34830]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T22:08:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:19 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:20 smithi092 ceph-mon[31173]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:20 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/1725210751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:08:21.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:20 smithi153 ceph-mon[35045]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:21.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:20 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/1725210751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:08:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:20 smithi110 ceph-mon[34830]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:20 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/1725210751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:08:23.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:22 smithi092 ceph-mon[31173]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:23.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:22 smithi153 ceph-mon[35045]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:22 smithi110 ceph-mon[34830]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:25.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:24 smithi092 ceph-mon[31173]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:25.019 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:24 smithi110 ceph-mon[34830]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:25.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:24 smithi153 ceph-mon[35045]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:26.977 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:26 smithi153 ceph-mon[35045]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:27.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:26 smithi092 ceph-mon[31173]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:26 smithi110 ceph-mon[34830]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:29.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:28 smithi092 ceph-mon[31173]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:29.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:08:29.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:29.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:28 smithi153 ceph-mon[35045]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:08:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:28 smithi110 ceph-mon[34830]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:08:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:29.984 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:29 smithi153 ceph-mon[35045]: Deploying daemon osd.5 on smithi153 2024-09-11T22:08:30.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:29 smithi092 ceph-mon[31173]: Deploying daemon osd.5 on smithi153 2024-09-11T22:08:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:29 smithi110 ceph-mon[34830]: Deploying daemon osd.5 on smithi153 2024-09-11T22:08:31.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:30 smithi092 ceph-mon[31173]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:31.020 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:30 smithi153 ceph-mon[35045]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:30 smithi110 ceph-mon[34830]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:33.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:32 smithi092 ceph-mon[31173]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:32 smithi153 ceph-mon[35045]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:32 smithi110 ceph-mon[34830]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:34.877 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:34 smithi153 ceph-mon[35045]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:34.877 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:34.877 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:34.877 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:34 smithi092 ceph-mon[31173]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:34 smithi110 ceph-mon[34830]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:37.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:36 smithi092 ceph-mon[31173]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:37.031 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:36 smithi153 ceph-mon[35045]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:36 smithi110 ceph-mon[34830]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:38.501 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.501 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:38.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:38.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:38 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.502 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:38 smithi153 ceph-mon[35045]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:38.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:38.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:38.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:38 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:38 smithi092 ceph-mon[31173]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:38.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.540+0000 7fe8fcff9640 1 -- 172.21.15.153:0/1372012606 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe900062630 con 0x7fe8e80a2bb0 2024-09-11T22:08:38.540 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 5 on host 'smithi153' 2024-09-11T22:08:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe8e80a2bb0 msgr2=0x7fe8e80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe8e80a2bb0 0x7fe8e80a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000c00 tx=0x7fe8f40314a0 comp rx=0 tx=0).stop 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900101c00 msgr2=0x7fe900197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900101c00 0x7fe900197e10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000eaa0 tx=0x7fe8f000ef60 comp rx=0 tx=0).stop 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 shutdown_connections 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe8e80a2bb0 0x7fe8e80a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9001037c0 0x7fe90019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe900102e00 0x7fe900198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 --2- 172.21.15.153:0/1372012606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe900101c00 0x7fe900197e10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 >> 172.21.15.153:0/1372012606 conn(0x7fe9000fd390 msgr2=0x7fe9001061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 shutdown_connections 2024-09-11T22:08:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:08:38.542+0000 7fe905b5f640 1 -- 172.21.15.153:0/1372012606 wait complete. 2024-09-11T22:08:38.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:38.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:38.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:38 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:38.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:38 smithi110 ceph-mon[34830]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:39.240 DEBUG:teuthology.orchestra.run.smithi153:osd.5> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T22:08:39.242 INFO:tasks.cephadm:Deploying osd.6 on smithi092 with /dev/vg_nvme/lv_2... 2024-09-11T22:08:39.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:08:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:08:39.518 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:39 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:39.518 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:39 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:39.518 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:39 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:39.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:39 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:39.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:39 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:39.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:39 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:39.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:39 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:39.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:39 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:39.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:39 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:40.445 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:40 smithi153 ceph-mon[35045]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:40.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:40 smithi092 ceph-mon[31173]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:40 smithi110 ceph-mon[34830]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:40.731 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:08:40 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:08:40.472+0000 7f0440dfd640 -1 osd.5 0 log_to_monitors true 2024-09-11T22:08:41.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:41 smithi092 ceph-mon[31173]: from='osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:08:41.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:41 smithi092 ceph-mon[31173]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:08:41.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:41 smithi153 ceph-mon[35045]: from='osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:08:41.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:41 smithi153 ceph-mon[35045]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:08:41.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:41 smithi110 ceph-mon[34830]: from='osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:08:41.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:41 smithi110 ceph-mon[34830]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:08:42.009 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:08:42.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-09-11T22:08:42.294 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: osdmap e38: 6 total, 5 up, 6 in 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:42.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:42.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:42.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:42.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.552 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:08:42.552 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: osdmap e38: 6 total, 5 up, 6 in 2024-09-11T22:08:42.552 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:42.552 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:42.553 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:08:42.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: osdmap e38: 6 total, 5 up, 6 in 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:08:42.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:42.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 -- 172.21.15.92:0/1563550841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c1037c0 msgr2=0x7f815c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 --2- 172.21.15.92:0/1563550841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c1037c0 0x7f815c10a050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8154009ca0 tx=0x7f815402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 -- 172.21.15.92:0/1563550841 shutdown_connections 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 --2- 172.21.15.92:0/1563550841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c1037c0 0x7f815c10a050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 --2- 172.21.15.92:0/1563550841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c102e00 0x7f815c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 --2- 172.21.15.92:0/1563550841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c101c00 0x7f815c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 -- 172.21.15.92:0/1563550841 >> 172.21.15.92:0/1563550841 conn(0x7f815c0fd390 msgr2=0x7f815c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 -- 172.21.15.92:0/1563550841 shutdown_connections 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.760+0000 7f8164048640 1 -- 172.21.15.92:0/1563550841 wait complete. 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 Processor -- start 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 -- start start 2024-09-11T22:08:42.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c101c00 0x7f815c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c102e00 0x7f815c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c1037c0 0x7f815c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f815c10d730 con 0x7f815c101c00 2024-09-11T22:08:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f815c10d5b0 con 0x7f815c102e00 2024-09-11T22:08:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8164048640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f815c10d8b0 con 0x7f815c1037c0 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8161de4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c101c00 0x7f815c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8161de4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c101c00 0x7f815c197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33100/0 (socket says 172.21.15.92:33100) 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.761+0000 7f8161de4640 1 -- 172.21.15.92:0/3062455454 learned_addr learned my addr 172.21.15.92:0/3062455454 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f81615e3640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c102e00 0x7f815c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8161de4640 1 -- 172.21.15.92:0/3062455454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c1037c0 msgr2=0x7f815c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8161de4640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c1037c0 0x7f815c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8161de4640 1 -- 172.21.15.92:0/3062455454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c102e00 msgr2=0x7f815c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8161de4640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c102e00 0x7f815c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8161de4640 1 -- 172.21.15.92:0/3062455454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c19f960 con 0x7f815c101c00 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f81615e3640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c102e00 0x7f815c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8161de4640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c101c00 0x7f815c197d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f814c00c970 tx=0x7f814c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8152ffd640 1 -- 172.21.15.92:0/3062455454 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c007c10 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c19fc50 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8152ffd640 1 -- 172.21.15.92:0/3062455454 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f814c007db0 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8152ffd640 1 -- 172.21.15.92:0/3062455454 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c00f870 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.762+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c1a0190 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.763+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c108f50 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.764+0000 7f8152ffd640 1 -- 172.21.15.92:0/3062455454 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f814c00fa10 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.765+0000 7f8152ffd640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81480a2b80 0x7f81480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.765+0000 7f81615e3640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81480a2b80 0x7f81480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.765+0000 7f8152ffd640 1 -- 172.21.15.92:0/3062455454 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4253+0+0 (secure 0 0 0) 0x7f814c0ce440 con 0x7f815c101c00 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.765+0000 7f81615e3640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81480a2b80 0x7f81480a5040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81580058c0 tx=0x7f8158005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.769+0000 7f8152ffd640 1 -- 172.21.15.92:0/3062455454 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f814c09c410 con 0x7f815c101c00 2024-09-11T22:08:42.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:08:42.946+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f815c062630 con 0x7f81480a2b80 2024-09-11T22:08:43.039 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:08:42.666+0000 7f043cd80640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T22:08:43.039 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:08:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:08:42.673+0000 7f0434ba0640 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi153 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: osdmap e39: 6 total, 5 up, 6 in 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='osd.5 ' entity='osd.5' 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:08:43.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:43.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] boot 2024-09-11T22:08:43.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T22:08:43.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:43 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.579 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi153 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: osdmap e39: 6 total, 5 up, 6 in 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='osd.5 ' entity='osd.5' 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] boot 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T22:08:43.580 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi153 2024-09-11T22:08:43.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-11T22:08:43.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:08:43.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: osdmap e39: 6 total, 5 up, 6 in 2024-09-11T22:08:43.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:43.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='osd.5 ' entity='osd.5' 2024-09-11T22:08:43.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:08:43.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:08:43.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:43.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: osd.5 [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] boot 2024-09-11T22:08:43.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T22:08:43.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:43 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:08:44.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:44 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:08:44.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:44 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:08:44.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:44 smithi092 ceph-mon[31173]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:08:44.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:44 smithi092 ceph-mon[31173]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:44.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:44 smithi092 ceph-mon[31173]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T22:08:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:44 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:08:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:44 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:08:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:44 smithi153 ceph-mon[35045]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:08:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:44 smithi153 ceph-mon[35045]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:44 smithi153 ceph-mon[35045]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T22:08:44.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:44 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:08:44.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:44 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:08:44.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:44 smithi110 ceph-mon[34830]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:08:44.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:44 smithi110 ceph-mon[34830]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:44.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:44 smithi110 ceph-mon[34830]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T22:08:45.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:45 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4238029757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "647ec3ad-3258-47f3-9c07-f242cda2ec4d"}]: dispatch 2024-09-11T22:08:45.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:45 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4238029757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "647ec3ad-3258-47f3-9c07-f242cda2ec4d"}]': finished 2024-09-11T22:08:45.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:45 smithi153 ceph-mon[35045]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:08:45.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:45 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:45.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:45 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4238029757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "647ec3ad-3258-47f3-9c07-f242cda2ec4d"}]: dispatch 2024-09-11T22:08:45.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:45 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4238029757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "647ec3ad-3258-47f3-9c07-f242cda2ec4d"}]': finished 2024-09-11T22:08:45.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:45 smithi110 ceph-mon[34830]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:08:45.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:45 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:45.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:45 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4238029757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "647ec3ad-3258-47f3-9c07-f242cda2ec4d"}]: dispatch 2024-09-11T22:08:45.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:45 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4238029757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "647ec3ad-3258-47f3-9c07-f242cda2ec4d"}]': finished 2024-09-11T22:08:45.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:45 smithi092 ceph-mon[31173]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:08:45.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:45 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:46.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:46 smithi153 ceph-mon[35045]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:46.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:46 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4250641158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:08:46.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:46 smithi153 ceph-mon[35045]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T22:08:46.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:46 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:46.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:46 smithi110 ceph-mon[34830]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:46.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:46 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4250641158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:08:46.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:46 smithi110 ceph-mon[34830]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T22:08:46.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:46 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:46.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:46 smithi092 ceph-mon[31173]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:46.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:46 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4250641158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:08:46.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:46 smithi092 ceph-mon[31173]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T22:08:46.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:46 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:48 smithi110 ceph-mon[34830]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T22:08:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:48 smithi110 ceph-mon[34830]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:48.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:48 smithi092 ceph-mon[31173]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T22:08:48.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:48.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:48 smithi092 ceph-mon[31173]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:48.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:48 smithi153 ceph-mon[35045]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T22:08:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:08:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:48 smithi153 ceph-mon[35045]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:08:50.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:50 smithi092 ceph-mon[31173]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-11T22:08:50.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:50 smithi110 ceph-mon[34830]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-11T22:08:51.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:50 smithi153 ceph-mon[35045]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-11T22:08:52.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:52 smithi110 ceph-mon[34830]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T22:08:53.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:52 smithi092 ceph-mon[31173]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T22:08:53.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:52 smithi153 ceph-mon[35045]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T22:08:54.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:54 smithi110 ceph-mon[34830]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:08:54.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:08:54.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:54.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:54 smithi110 ceph-mon[34830]: Deploying daemon osd.6 on smithi092 2024-09-11T22:08:54.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:54 smithi092 ceph-mon[31173]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:08:54.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:08:54.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:54.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:54 smithi092 ceph-mon[31173]: Deploying daemon osd.6 on smithi092 2024-09-11T22:08:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:54 smithi153 ceph-mon[35045]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:08:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:08:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:08:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:54 smithi153 ceph-mon[35045]: Deploying daemon osd.6 on smithi092 2024-09-11T22:08:56.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:56 smithi110 ceph-mon[34830]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T22:08:56.986 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:56 smithi092 ceph-mon[31173]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T22:08:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:56 smithi153 ceph-mon[35045]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T22:08:58.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:58 smithi110 ceph-mon[34830]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T22:08:59.016 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:58 smithi092 ceph-mon[31173]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T22:08:59.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:58 smithi153 ceph-mon[35045]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T22:08:59.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:59 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:08:59.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:59 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:08:59.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:08:59 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:59 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:59 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:08:59 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:59 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:59 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:08:59 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:00.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:00 smithi110 ceph-mon[34830]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:09:00.986 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:00 smithi092 ceph-mon[31173]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:09:01.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:00 smithi153 ceph-mon[35045]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:09:02.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:02 smithi110 ceph-mon[34830]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:02.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:02.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:02.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:02.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:02.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:02 smithi092 ceph-mon[31173]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:02 smithi153 ceph-mon[35045]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.385+0000 7f8152ffd640 1 -- 172.21.15.92:0/3062455454 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f815c062630 con 0x7f81480a2b80 2024-09-11T22:09:03.387 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 6 on host 'smithi092' 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.387+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81480a2b80 msgr2=0x7f81480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.387+0000 7f8164048640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81480a2b80 0x7f81480a5040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81580058c0 tx=0x7f8158005d30 comp rx=0 tx=0).stop 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.387+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c101c00 msgr2=0x7f815c197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.387+0000 7f8164048640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c101c00 0x7f815c197d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f814c00c970 tx=0x7f814c00ce30 comp rx=0 tx=0).stop 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 shutdown_connections 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81480a2b80 0x7f81480a5040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c1037c0 0x7f815c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c102e00 0x7f815c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 --2- 172.21.15.92:0/3062455454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815c101c00 0x7f815c197d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 >> 172.21.15.92:0/3062455454 conn(0x7f815c0fd390 msgr2=0x7f815c106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 shutdown_connections 2024-09-11T22:09:03.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:03.388+0000 7f8164048640 1 -- 172.21.15.92:0/3062455454 wait complete. 2024-09-11T22:09:03.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:03 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:03.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:03 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:03 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.995 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:03 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:03.996 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:03 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:03.996 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:03 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:04.021 DEBUG:teuthology.orchestra.run.smithi092:osd.6> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T22:09:04.024 INFO:tasks.cephadm:Deploying osd.7 on smithi110 with /dev/vg_nvme/lv_2... 2024-09-11T22:09:04.024 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:09:04.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:03 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:04.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:03 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:04.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:03 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:04.278 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:09:04.903 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:04 smithi092 ceph-mon[31173]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:04.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:04 smithi110 ceph-mon[34830]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:05.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:04 smithi153 ceph-mon[35045]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:05.447 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:09:05 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:09:05.214+0000 7ff811087640 -1 osd.6 0 log_to_monitors true 2024-09-11T22:09:06.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:05 smithi092 ceph-mon[31173]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:09:06.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:05 smithi153 ceph-mon[35045]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:09:06.100 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:05 smithi110 ceph-mon[34830]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:09:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:09:06.865 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-09-11T22:09:06.890 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:06 smithi110 ceph-mon[34830]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:06.890 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:06 smithi110 ceph-mon[34830]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:09:06.891 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:06 smithi110 ceph-mon[34830]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T22:09:06.891 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:06 smithi110 ceph-mon[34830]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:09:06.891 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:06.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:06 smithi092 ceph-mon[31173]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:06.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:06 smithi092 ceph-mon[31173]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:09:06.923 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:06 smithi092 ceph-mon[31173]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T22:09:06.923 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:06 smithi092 ceph-mon[31173]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:09:06.923 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:06 smithi153 ceph-mon[35045]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:06 smithi153 ceph-mon[35045]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:09:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:06 smithi153 ceph-mon[35045]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T22:09:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:06 smithi153 ceph-mon[35045]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:09:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:07.125 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:09:07.268 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:09:07.113+0000 7ff80d81e640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T22:09:07.268 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:09:07.121+0000 7ff804e2a640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/3804879326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4100ad0 msgr2=0x7ff3e4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/3804879326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4100ad0 0x7ff3e4100ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009900 tx=0x7ff3d802a410 comp rx=0 tx=0).stop 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/3804879326 shutdown_connections 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/3804879326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3e4102690 0x7ff3e4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/3804879326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e4101cd0 0x7ff3e4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/3804879326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4100ad0 0x7ff3e4100ed0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/3804879326 >> 172.21.15.110:0/3804879326 conn(0x7ff3e40fc280 msgr2=0x7ff3e40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/3804879326 shutdown_connections 2024-09-11T22:09:07.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.580+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/3804879326 wait complete. 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 Processor -- start 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 -- start start 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e4100ad0 0x7ff3e406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3e4101cd0 0x7ff3e419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4102690 0x7ff3e419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e410c660 con 0x7ff3e4101cd0 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e410c4e0 con 0x7ff3e4102690 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ebdb2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e410c7e0 con 0x7ff3e4100ad0 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4102690 0x7ff3e419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4102690 0x7ff3e419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51998/0 (socket says 172.21.15.110:51998) 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 -- 172.21.15.110:0/1506182827 learned_addr learned my addr 172.21.15.110:0/1506182827 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3e934d640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3e4101cd0 0x7ff3e419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3e9b4e640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e4100ad0 0x7ff3e406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 -- 172.21.15.110:0/1506182827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e4100ad0 msgr2=0x7ff3e406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:07.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e4100ad0 0x7ff3e406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 -- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3e4101cd0 msgr2=0x7ff3e419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3e4101cd0 0x7ff3e419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.581+0000 7ff3ea34f640 1 -- 172.21.15.110:0/1506182827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e419fb00 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.582+0000 7ff3ea34f640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4102690 0x7ff3e419f1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00eaa0 tx=0x7ff3dc00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.582+0000 7ff3d6ffd640 1 -- 172.21.15.110:0/1506182827 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc004d20 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.582+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e419fd90 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.582+0000 7ff3d6ffd640 1 -- 172.21.15.110:0/1506182827 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3dc005020 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.582+0000 7ff3d6ffd640 1 -- 172.21.15.110:0/1506182827 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0116b0 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.582+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e41a0350 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.583+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e4107f90 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.584+0000 7ff3d6ffd640 1 -- 172.21.15.110:0/1506182827 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff3dc0051d0 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.584+0000 7ff3d6ffd640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3d00a2b80 0x7ff3d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.585+0000 7ff3d6ffd640 1 -- 172.21.15.110:0/1506182827 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4685+0+0 (secure 0 0 0) 0x7ff3dc0ce3a0 con 0x7ff3e4102690 2024-09-11T22:09:07.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.585+0000 7ff3e9b4e640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3d00a2b80 0x7ff3d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:07.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.585+0000 7ff3e9b4e640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3d00a2b80 0x7ff3d00a5040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8004770 tx=0x7ff3d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:09:07.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.589+0000 7ff3d6ffd640 1 -- 172.21.15.110:0/1506182827 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3dc016030 con 0x7ff3e4102690 2024-09-11T22:09:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:07.764+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff3e4062630 con 0x7ff3d00a2b80 2024-09-11T22:09:07.765 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:09:07.765 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: osdmap e46: 7 total, 6 up, 7 in 2024-09-11T22:09:07.765 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:07.765 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:07.765 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:07.765 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:07.766 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:07 smithi110 ceph-mon[34830]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' 2024-09-11T22:09:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:09:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: osdmap e46: 7 total, 6 up, 7 in 2024-09-11T22:09:08.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:07 smithi092 ceph-mon[31173]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: osdmap e46: 7 total, 6 up, 7 in 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:08.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:07 smithi153 ceph-mon[35045]: from='osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050]' entity='osd.6' 2024-09-11T22:09:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:09:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi092 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] boot 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T22:09:09.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi092 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] boot 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T22:09:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:09.147 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:09:09.147 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:09:09.147 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi092 2024-09-11T22:09:09.147 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-11T22:09:09.147 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:09:09.147 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:09.148 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:09:09.148 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:09.148 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:09.148 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:09.148 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: osd.6 [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] boot 2024-09-11T22:09:09.148 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T22:09:09.148 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:09:10.421 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:10 smithi110 ceph-mon[34830]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T22:09:10.421 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:10 smithi110 ceph-mon[34830]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:10.421 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:10 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/3743166395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]: dispatch 2024-09-11T22:09:10.421 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:10 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]: dispatch 2024-09-11T22:09:10.421 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:10 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]': finished 2024-09-11T22:09:10.421 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:10 smithi110 ceph-mon[34830]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T22:09:10.421 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:10.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:10 smithi092 ceph-mon[31173]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T22:09:10.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:10 smithi092 ceph-mon[31173]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:10.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:10 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/3743166395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]: dispatch 2024-09-11T22:09:10.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:10 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]: dispatch 2024-09-11T22:09:10.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:10 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]': finished 2024-09-11T22:09:10.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:10 smithi092 ceph-mon[31173]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T22:09:10.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:10.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:10 smithi153 ceph-mon[35045]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T22:09:10.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:10 smithi153 ceph-mon[35045]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:10.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:10 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/3743166395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]: dispatch 2024-09-11T22:09:10.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:10 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]: dispatch 2024-09-11T22:09:10.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:10 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb"}]': finished 2024-09-11T22:09:10.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:10 smithi153 ceph-mon[35045]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T22:09:10.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:11.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:11 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/2943621842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:09:11.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:11 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/2943621842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:09:11.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:11 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/2943621842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:09:12.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:12 smithi110 ceph-mon[34830]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:12.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:12 smithi092 ceph-mon[31173]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:12.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:12 smithi153 ceph-mon[35045]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:14.951 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:14 smithi110 ceph-mon[34830]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:14 smithi092 ceph-mon[31173]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:15.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:14 smithi153 ceph-mon[35045]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:16.705 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:16 smithi110 ceph-mon[34830]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:16 smithi092 ceph-mon[31173]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:16 smithi153 ceph-mon[35045]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:18.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:18 smithi110 ceph-mon[34830]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:18.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:09:18.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:18 smithi092 ceph-mon[31173]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:09:19.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:19.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:18 smithi153 ceph-mon[35045]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:09:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:19.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:19 smithi110 ceph-mon[34830]: Deploying daemon osd.7 on smithi110 2024-09-11T22:09:20.021 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:19 smithi092 ceph-mon[31173]: Deploying daemon osd.7 on smithi110 2024-09-11T22:09:20.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:19 smithi153 ceph-mon[35045]: Deploying daemon osd.7 on smithi110 2024-09-11T22:09:20.927 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:20 smithi110 ceph-mon[34830]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:20 smithi092 ceph-mon[31173]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:21.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:20 smithi153 ceph-mon[35045]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:22.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:22 smithi110 ceph-mon[34830]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:23.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:22 smithi092 ceph-mon[31173]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:23.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:22 smithi153 ceph-mon[35045]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:24.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:24 smithi110 ceph-mon[34830]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:24.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:24 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:24.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:24 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:24.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:24 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:25.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:24 smithi092 ceph-mon[31173]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:25.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:24 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:25.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:24 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:25.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:24 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:25.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:24 smithi153 ceph-mon[35045]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:25.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:24 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:25.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:24 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:25.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:24 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:26.882 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:26 smithi110 ceph-mon[34830]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:27.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:26 smithi092 ceph-mon[31173]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:27.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:26 smithi153 ceph-mon[35045]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:28.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.446+0000 7ff3d6ffd640 1 -- 172.21.15.110:0/1506182827 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff3e4062630 con 0x7ff3d00a2b80 2024-09-11T22:09:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 7 on host 'smithi110' 2024-09-11T22:09:28.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.448+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3d00a2b80 msgr2=0x7ff3d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:28.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.448+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3d00a2b80 0x7ff3d00a5040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8004770 tx=0x7ff3d8000bc0 comp rx=0 tx=0).stop 2024-09-11T22:09:28.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4102690 msgr2=0x7ff3e419f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4102690 0x7ff3e419f1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00eaa0 tx=0x7ff3dc00eea0 comp rx=0 tx=0).stop 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 shutdown_connections 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3d00a2b80 0x7ff3d00a5040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e4102690 0x7ff3e419f1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3e4101cd0 0x7ff3e419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 --2- 172.21.15.110:0/1506182827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e4100ad0 0x7ff3e406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 >> 172.21.15.110:0/1506182827 conn(0x7ff3e40fc280 msgr2=0x7ff3e4105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 shutdown_connections 2024-09-11T22:09:28.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:09:28.449+0000 7ff3ebdb2640 1 -- 172.21.15.110:0/1506182827 wait complete. 2024-09-11T22:09:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:28 smithi110 ceph-mon[34830]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:28.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:28.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:28.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:28 smithi092 ceph-mon[31173]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:28.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:28 smithi153 ceph-mon[35045]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:29.068 DEBUG:teuthology.orchestra.run.smithi110:osd.7> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T22:09:29.111 INFO:tasks.cephadm:Deploying osd.8 on smithi153 with /dev/vg_nvme/lv_2... 2024-09-11T22:09:29.111 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:09:29.365 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:09:29.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:29.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:29.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:29.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:29.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:29.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:29.786 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:29.786 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:29.786 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:30.291 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:09:30 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:09:30.199+0000 7fb5aacc1640 -1 osd.7 0 log_to_monitors true 2024-09-11T22:09:30.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:30 smithi153 ceph-mon[35045]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:30.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:30 smithi153 ceph-mon[35045]: from='osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:09:30.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:30 smithi153 ceph-mon[35045]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:09:30.580 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:30 smithi110 ceph-mon[34830]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:30.580 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:30 smithi110 ceph-mon[34830]: from='osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:09:30.580 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:30 smithi110 ceph-mon[34830]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:09:30.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:30 smithi092 ceph-mon[31173]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:30.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:30 smithi092 ceph-mon[31173]: from='osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:09:30.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:30 smithi092 ceph-mon[31173]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:09:31.577 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:31 smithi153 ceph-mon[35045]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:09:31.577 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:31 smithi153 ceph-mon[35045]: from='osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:09:31.577 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:31 smithi153 ceph-mon[35045]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T22:09:31.577 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:31 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:31.577 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:31 smithi153 ceph-mon[35045]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:09:31.670 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:31 smithi110 ceph-mon[34830]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:09:31.670 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:31 smithi110 ceph-mon[34830]: from='osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:09:31.670 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:31 smithi110 ceph-mon[34830]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T22:09:31.670 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:31 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:31.670 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:31 smithi110 ceph-mon[34830]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:09:31.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:31 smithi092 ceph-mon[31173]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:09:31.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:31 smithi092 ceph-mon[31173]: from='osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:09:31.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:31 smithi092 ceph-mon[31173]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T22:09:31.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:31 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:31.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:31 smithi092 ceph-mon[31173]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:09:31.950 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:09:31 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:09:31.867+0000 7fb5a6c44640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T22:09:31.951 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:09:31 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:09:31.875+0000 7fb59ea64640 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:09:32.102 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:09:32.129 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2024-09-11T22:09:32.380 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:09:32.708 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:09:32.708 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T22:09:32.708 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.708 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='osd.7 ' entity='osd.7' 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.709 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='osd.7 ' entity='osd.7' 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='osd.7 ' entity='osd.7' 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:32.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:32.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:32.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.910+0000 7fc86c47d640 1 -- 172.21.15.153:0/1409538608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 msgr2=0x7fc864102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.910+0000 7fc86c47d640 1 --2- 172.21.15.153:0/1409538608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864102190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc854009900 tx=0x7fc85402a410 comp rx=0 tx=0).stop 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- 172.21.15.153:0/1409538608 shutdown_connections 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 --2- 172.21.15.153:0/1409538608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8641026d0 0x7fc864108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 --2- 172.21.15.153:0/1409538608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864102190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 --2- 172.21.15.153:0/1409538608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc864100b10 0x7fc864100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- 172.21.15.153:0/1409538608 >> 172.21.15.153:0/1409538608 conn(0x7fc8640fc2a0 msgr2=0x7fc8640fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- 172.21.15.153:0/1409538608 shutdown_connections 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- 172.21.15.153:0/1409538608 wait complete. 2024-09-11T22:09:32.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 Processor -- start 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- start start 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc864100b10 0x7fc864196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8641026d0 0x7fc86419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc86410c620 con 0x7fc864100b10 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc86410c4a0 con 0x7fc8641026d0 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.911+0000 7fc86c47d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc86410c7a0 con 0x7fc864101d10 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42602/0 (socket says 172.21.15.153:42602) 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 -- 172.21.15.153:0/3720558581 learned_addr learned my addr 172.21.15.153:0/3720558581 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:09:32.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc86a219640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc864100b10 0x7fc864196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc86aa1a640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8641026d0 0x7fc86419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 -- 172.21.15.153:0/3720558581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8641026d0 msgr2=0x7fc86419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8641026d0 0x7fc86419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 -- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc864100b10 msgr2=0x7fc864196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc864100b10 0x7fc864196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 -- 172.21.15.153:0/3720558581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86419e910 con 0x7fc864101d10 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc86aa1a640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8641026d0 0x7fc86419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc869a18640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864197230 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc854005940 tx=0x7fc854004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc85b7fe640 1 -- 172.21.15.153:0/3720558581 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85403f030 con 0x7fc864101d10 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc85b7fe640 1 -- 172.21.15.153:0/3720558581 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc854003cb0 con 0x7fc864101d10 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86419eba0 con 0x7fc864101d10 2024-09-11T22:09:32.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc85b7fe640 1 -- 172.21.15.153:0/3720558581 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854035070 con 0x7fc864101d10 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.912+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86419f080 con 0x7fc864101d10 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.913+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc864107f60 con 0x7fc864101d10 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.914+0000 7fc85b7fe640 1 -- 172.21.15.153:0/3720558581 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc854042030 con 0x7fc864101d10 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.915+0000 7fc85b7fe640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc8500a2b80 0x7fc8500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.915+0000 7fc85b7fe640 1 -- 172.21.15.153:0/3720558581 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5430+0+0 (secure 0 0 0) 0x7fc8540ef280 con 0x7fc864101d10 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.915+0000 7fc86a219640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc8500a2b80 0x7fc8500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.916+0000 7fc86a219640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc8500a2b80 0x7fc8500a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8600058c0 tx=0x7fc860005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:09:32.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:32.918+0000 7fc85b7fe640 1 -- 172.21.15.153:0/3720558581 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8540bcd20 con 0x7fc864101d10 2024-09-11T22:09:33.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:33.095+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc864062630 con 0x7fc8500a2b80 2024-09-11T22:09:33.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:09:33.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:09:33.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi110 2024-09-11T22:09:33.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-11T22:09:33.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] boot 2024-09-11T22:09:33.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T22:09:33.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:33.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:33.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:33.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:33 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:33.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:09:33.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi110 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] boot 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:33 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi110 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: osd.7 [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] boot 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:33.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:33 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:34.726 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:34 smithi153 ceph-mon[35045]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:09:34.726 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:34 smithi153 ceph-mon[35045]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T22:09:34.726 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:34 smithi153 ceph-mon[35045]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:34.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:34 smithi092 ceph-mon[31173]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:09:34.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:34 smithi092 ceph-mon[31173]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T22:09:34.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:34 smithi092 ceph-mon[31173]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:34 smithi110 ceph-mon[34830]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:09:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:34 smithi110 ceph-mon[34830]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T22:09:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:34 smithi110 ceph-mon[34830]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:35.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:35 smithi092 ceph-mon[31173]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T22:09:35.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:35 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/1846135403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]: dispatch 2024-09-11T22:09:35.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:35 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]: dispatch 2024-09-11T22:09:35.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:35 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]': finished 2024-09-11T22:09:35.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:35 smithi092 ceph-mon[31173]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T22:09:35.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:35 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:35.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:35 smithi153 ceph-mon[35045]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T22:09:35.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:35 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/1846135403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]: dispatch 2024-09-11T22:09:35.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:35 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]: dispatch 2024-09-11T22:09:35.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:35 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]': finished 2024-09-11T22:09:35.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:35 smithi153 ceph-mon[35045]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T22:09:35.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:35 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:35.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:35 smithi110 ceph-mon[34830]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T22:09:35.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:35 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/1846135403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]: dispatch 2024-09-11T22:09:35.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:35 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]: dispatch 2024-09-11T22:09:35.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:35 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37cde37b-22c1-4525-9d66-ca50a29c7060"}]': finished 2024-09-11T22:09:35.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:35 smithi110 ceph-mon[34830]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T22:09:35.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:35 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:36.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:36 smithi092 ceph-mon[31173]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:36.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:36 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/1496130853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:09:36.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:36 smithi092 ceph-mon[31173]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T22:09:36.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:36 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:36.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:36 smithi153 ceph-mon[35045]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:36.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:36 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/1496130853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:09:36.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:36 smithi153 ceph-mon[35045]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T22:09:36.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:36 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:36.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:36 smithi110 ceph-mon[34830]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:36.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:36 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/1496130853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:09:36.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:36 smithi110 ceph-mon[34830]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T22:09:36.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:36 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:38.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:38 smithi110 ceph-mon[34830]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:39.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:38 smithi092 ceph-mon[31173]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:39.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:38 smithi153 ceph-mon[35045]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:40.917 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:40 smithi153 ceph-mon[35045]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:40.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:40 smithi110 ceph-mon[34830]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:41.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:40 smithi092 ceph-mon[31173]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:42.888 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:42 smithi153 ceph-mon[35045]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:42.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:42 smithi110 ceph-mon[34830]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:42 smithi092 ceph-mon[31173]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:44.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:44 smithi110 ceph-mon[34830]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:44.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:09:44.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:44 smithi092 ceph-mon[31173]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:09:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:45.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:44 smithi153 ceph-mon[35045]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:09:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:45.908 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:45 smithi153 ceph-mon[35045]: Deploying daemon osd.8 on smithi153 2024-09-11T22:09:46.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:45 smithi092 ceph-mon[31173]: Deploying daemon osd.8 on smithi153 2024-09-11T22:09:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:45 smithi110 ceph-mon[34830]: Deploying daemon osd.8 on smithi153 2024-09-11T22:09:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:46 smithi092 ceph-mon[31173]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:47.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:46 smithi153 ceph-mon[35045]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:46 smithi110 ceph-mon[34830]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:49.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:48 smithi092 ceph-mon[31173]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:49.037 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:48 smithi153 ceph-mon[35045]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:48 smithi110 ceph-mon[34830]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:50 smithi092 ceph-mon[31173]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:51.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:50 smithi153 ceph-mon[35045]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:50 smithi110 ceph-mon[34830]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:52.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:52 smithi153 ceph-mon[35045]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:53.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:52 smithi092 ceph-mon[31173]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:52 smithi110 ceph-mon[34830]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:53.336 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 8 on host 'smithi153' 2024-09-11T22:09:53.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.334+0000 7fc85b7fe640 1 -- 172.21.15.153:0/3720558581 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc864062630 con 0x7fc8500a2b80 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc8500a2b80 msgr2=0x7fc8500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc8500a2b80 0x7fc8500a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8600058c0 tx=0x7fc860005850 comp rx=0 tx=0).stop 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 msgr2=0x7fc864197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864197230 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc854005940 tx=0x7fc854004430 comp rx=0 tx=0).stop 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 shutdown_connections 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc8500a2b80 0x7fc8500a5040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8641026d0 0x7fc86419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc864101d10 0x7fc864197230 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 --2- 172.21.15.153:0/3720558581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc864100b10 0x7fc864196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:53.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 >> 172.21.15.153:0/3720558581 conn(0x7fc8640fc2a0 msgr2=0x7fc864105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:09:53.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 shutdown_connections 2024-09-11T22:09:53.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:09:53.336+0000 7fc86c47d640 1 -- 172.21.15.153:0/3720558581 wait complete. 2024-09-11T22:09:54.044 DEBUG:teuthology.orchestra.run.smithi153:osd.8> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T22:09:54.047 INFO:tasks.cephadm:Deploying osd.9 on smithi092 with /dev/vg_nvme/lv_1... 2024-09-11T22:09:54.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:09:54.300 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:09:54.580 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.580 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.580 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:54 smithi153 ceph-mon[35045]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:54.580 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.580 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.643 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.644 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.644 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:54 smithi092 ceph-mon[31173]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:54.644 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.644 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:54 smithi110 ceph-mon[34830]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:54.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:54.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:55.904 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:09:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:09:55.769+0000 7efe78c94640 -1 osd.8 0 log_to_monitors true 2024-09-11T22:09:56.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:56 smithi110 ceph-mon[34830]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:56.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:56 smithi110 ceph-mon[34830]: from='osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:09:56.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:56 smithi110 ceph-mon[34830]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:09:57.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:56 smithi092 ceph-mon[31173]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:57.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:56 smithi092 ceph-mon[31173]: from='osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:09:57.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:56 smithi092 ceph-mon[31173]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:09:57.020 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:09:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:56 smithi153 ceph-mon[35045]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:56 smithi153 ceph-mon[35045]: from='osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:09:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:56 smithi153 ceph-mon[35045]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:09:57.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-09-11T22:09:57.291 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:09:57.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 -- 172.21.15.92:0/4290244640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc100350 msgr2=0x7f7dfc10a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:57.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 --2- 172.21.15.92:0/4290244640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc100350 0x7f7dfc10a200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009ca0 tx=0x7f7df402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:09:57.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 -- 172.21.15.92:0/4290244640 shutdown_connections 2024-09-11T22:09:57.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 --2- 172.21.15.92:0/4290244640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc100350 0x7f7dfc10a200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:57.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 --2- 172.21.15.92:0/4290244640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dfc103be0 0x7f7dfc0ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 --2- 172.21.15.92:0/4290244640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc103210 0x7f7dfc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 -- 172.21.15.92:0/4290244640 >> 172.21.15.92:0/4290244640 conn(0x7f7dfc0fba40 msgr2=0x7f7dfc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 -- 172.21.15.92:0/4290244640 shutdown_connections 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.740+0000 7f7e0392f640 1 -- 172.21.15.92:0/4290244640 wait complete. 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 Processor -- start 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 -- start start 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc100350 0x7f7dfc102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dfc103210 0x7f7dfc100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc103be0 0x7f7dfc1014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc10d780 con 0x7f7dfc103be0 2024-09-11T22:09:57.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc10d600 con 0x7f7dfc103210 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e0392f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc10d900 con 0x7f7dfc100350 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e01ecc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc103be0 0x7f7dfc1014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e01ecc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc103be0 0x7f7dfc1014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34294/0 (socket says 172.21.15.92:34294) 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e016cb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc100350 0x7f7dfc102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e01ecc640 1 -- 172.21.15.92:0/2312549470 learned_addr learned my addr 172.21.15.92:0/2312549470 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.741+0000 7f7e00eca640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dfc103210 0x7f7dfc100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e01ecc640 1 -- 172.21.15.92:0/2312549470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc100350 msgr2=0x7f7dfc102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e01ecc640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc100350 0x7f7dfc102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e01ecc640 1 -- 172.21.15.92:0/2312549470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dfc103210 msgr2=0x7f7dfc100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e01ecc640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dfc103210 0x7f7dfc100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e01ecc640 1 -- 172.21.15.92:0/2312549470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc101d60 con 0x7f7dfc103be0 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e00eca640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dfc103210 0x7f7dfc100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:09:57.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e01ecc640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc103be0 0x7f7dfc1014a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7df4006010 tx=0x7f7df4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7df27fc640 1 -- 172.21.15.92:0/2312549470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df40317e0 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7df27fc640 1 -- 172.21.15.92:0/2312549470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df40044f0 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc1a1fa0 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7df27fc640 1 -- 172.21.15.92:0/2312549470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4030c00 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.742+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a2450 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.743+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc109250 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.744+0000 7f7df27fc640 1 -- 172.21.15.92:0/2312549470 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7df4004040 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.744+0000 7f7df27fc640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7de80a2b80 0x7f7de80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.745+0000 7f7e016cb640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7de80a2b80 0x7f7de80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.745+0000 7f7df27fc640 1 -- 172.21.15.92:0/2312549470 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5549+0+0 (secure 0 0 0) 0x7f7df40ef9d0 con 0x7f7dfc103be0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.745+0000 7f7e016cb640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7de80a2b80 0x7f7de80a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005fd0 tx=0x7f7df800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:09:57.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.749+0000 7f7df27fc640 1 -- 172.21.15.92:0/2312549470 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7df40bd400 con 0x7f7dfc103be0 2024-09-11T22:09:57.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:09:57.924+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7dfc062630 con 0x7f7de80a2b80 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: osdmap e57: 9 total, 8 up, 9 in 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:57.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: osdmap e57: 9 total, 8 up, 9 in 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:58.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:58.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: osdmap e57: 9 total, 8 up, 9 in 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:09:58.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:09:58.538 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:09:58.131+0000 7efe7542b640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T22:09:58.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:09:58.140+0000 7efe6ca37640 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:09:59.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi153 2024-09-11T22:09:59.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-11T22:09:59.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: osdmap e58: 9 total, 8 up, 9 in 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:58 smithi092 ceph-mon[31173]: from='osd.8 ' entity='osd.8' 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi153 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: osdmap e58: 9 total, 8 up, 9 in 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:58 smithi153 ceph-mon[35045]: from='osd.8 ' entity='osd.8' 2024-09-11T22:09:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi153 2024-09-11T22:09:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-11T22:09:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:09:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:09:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: osdmap e58: 9 total, 8 up, 9 in 2024-09-11T22:09:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:09:59.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:09:59.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:09:59.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:09:59.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:58 smithi110 ceph-mon[34830]: from='osd.8 ' entity='osd.8' 2024-09-11T22:10:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:59 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:10:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:59 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:10:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:59 smithi092 ceph-mon[31173]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:00.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:59 smithi092 ceph-mon[31173]: osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] boot 2024-09-11T22:10:00.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:59 smithi092 ceph-mon[31173]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T22:10:00.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:59 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:10:00.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:09:59 smithi092 ceph-mon[31173]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T22:10:00.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:59 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:10:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:59 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:10:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:59 smithi153 ceph-mon[35045]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:59 smithi153 ceph-mon[35045]: osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] boot 2024-09-11T22:10:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:59 smithi153 ceph-mon[35045]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T22:10:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:59 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:10:00.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:09:59 smithi153 ceph-mon[35045]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T22:10:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:59 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:10:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:59 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:10:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:59 smithi110 ceph-mon[34830]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:59 smithi110 ceph-mon[34830]: osd.8 [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] boot 2024-09-11T22:10:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:59 smithi110 ceph-mon[34830]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T22:10:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:59 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:10:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:09:59 smithi110 ceph-mon[34830]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T22:10:00.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:00 smithi092 ceph-mon[31173]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:10:00.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:00 smithi092 ceph-mon[31173]: overall HEALTH_OK 2024-09-11T22:10:00.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:00 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2036664549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c38dd0d9-39fb-4c67-9958-18eeb82ef314"}]: dispatch 2024-09-11T22:10:00.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:00 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2036664549' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c38dd0d9-39fb-4c67-9958-18eeb82ef314"}]': finished 2024-09-11T22:10:00.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:00 smithi092 ceph-mon[31173]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:10:00.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:00 smithi153 ceph-mon[35045]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:10:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:00 smithi153 ceph-mon[35045]: overall HEALTH_OK 2024-09-11T22:10:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:00 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2036664549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c38dd0d9-39fb-4c67-9958-18eeb82ef314"}]: dispatch 2024-09-11T22:10:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:00 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2036664549' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c38dd0d9-39fb-4c67-9958-18eeb82ef314"}]': finished 2024-09-11T22:10:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:00 smithi153 ceph-mon[35045]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:10:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:00 smithi110 ceph-mon[34830]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:10:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:00 smithi110 ceph-mon[34830]: overall HEALTH_OK 2024-09-11T22:10:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:00 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2036664549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c38dd0d9-39fb-4c67-9958-18eeb82ef314"}]: dispatch 2024-09-11T22:10:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:00 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2036664549' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c38dd0d9-39fb-4c67-9958-18eeb82ef314"}]': finished 2024-09-11T22:10:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:00 smithi110 ceph-mon[34830]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:10:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:01 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2349514573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:01 smithi092 ceph-mon[31173]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T22:10:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:02.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:01 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2349514573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:02.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:01 smithi153 ceph-mon[35045]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T22:10:02.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:01 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2349514573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:01 smithi110 ceph-mon[34830]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T22:10:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:02 smithi092 ceph-mon[31173]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:02 smithi092 ceph-mon[31173]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T22:10:03.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:03.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:02 smithi153 ceph-mon[35045]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:02 smithi153 ceph-mon[35045]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T22:10:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:02 smithi110 ceph-mon[34830]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:02 smithi110 ceph-mon[34830]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T22:10:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:04 smithi092 ceph-mon[31173]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:10:05.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:04 smithi153 ceph-mon[35045]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:10:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:04 smithi110 ceph-mon[34830]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:10:07.006 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:06 smithi092 ceph-mon[31173]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T22:10:07.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:06 smithi153 ceph-mon[35045]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T22:10:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:06 smithi110 ceph-mon[34830]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T22:10:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:08 smithi092 ceph-mon[31173]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T22:10:09.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:08 smithi153 ceph-mon[35045]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T22:10:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:08 smithi110 ceph-mon[34830]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T22:10:09.845 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:10:09.845 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:10.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:10:10.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:10:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:10.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:10 smithi092 ceph-mon[31173]: Deploying daemon osd.9 on smithi092 2024-09-11T22:10:10.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:10 smithi092 ceph-mon[31173]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T22:10:11.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:10 smithi153 ceph-mon[35045]: Deploying daemon osd.9 on smithi092 2024-09-11T22:10:11.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:10 smithi153 ceph-mon[35045]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T22:10:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:10 smithi110 ceph-mon[34830]: Deploying daemon osd.9 on smithi092 2024-09-11T22:10:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:10 smithi110 ceph-mon[34830]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T22:10:12.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:12 smithi092 ceph-mon[31173]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:10:13.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:12 smithi153 ceph-mon[35045]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:10:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:12 smithi110 ceph-mon[34830]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:10:15.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:14 smithi092 ceph-mon[31173]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:10:15.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:14 smithi153 ceph-mon[35045]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:10:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:14 smithi110 ceph-mon[34830]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:10:16.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:16.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:16.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:16.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:17.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:16 smithi153 ceph-mon[35045]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:17.128 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:16 smithi092 ceph-mon[31173]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:16 smithi110 ceph-mon[34830]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:18.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.805+0000 7f7df27fc640 1 -- 172.21.15.92:0/2312549470 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7dfc062630 con 0x7f7de80a2b80 2024-09-11T22:10:18.806 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 9 on host 'smithi092' 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7de80a2b80 msgr2=0x7f7de80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7de80a2b80 0x7f7de80a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005fd0 tx=0x7f7df800b390 comp rx=0 tx=0).stop 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc103be0 msgr2=0x7f7dfc1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc103be0 0x7f7dfc1014a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7df4006010 tx=0x7f7df4003ba0 comp rx=0 tx=0).stop 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 shutdown_connections 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7de80a2b80 0x7f7de80a5040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc103be0 0x7f7dfc1014a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dfc103210 0x7f7dfc100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.807+0000 7f7e0392f640 1 --2- 172.21.15.92:0/2312549470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dfc100350 0x7f7dfc102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.808+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 >> 172.21.15.92:0/2312549470 conn(0x7f7dfc0fba40 msgr2=0x7f7dfc106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:10:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.808+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 shutdown_connections 2024-09-11T22:10:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:10:18.808+0000 7f7e0392f640 1 -- 172.21.15.92:0/2312549470 wait complete. 2024-09-11T22:10:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:18 smithi092 ceph-mon[31173]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:19.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:19.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:18 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:18 smithi153 ceph-mon[35045]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:18 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:18 smithi110 ceph-mon[34830]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:18 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.513 DEBUG:teuthology.orchestra.run.smithi092:osd.9> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T22:10:19.515 INFO:tasks.cephadm:Deploying osd.10 on smithi110 with /dev/vg_nvme/lv_1... 2024-09-11T22:10:19.515 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:10:19.770 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:10:19.892 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:19 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:19.892 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:19 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:19.892 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:19 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:20.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:19 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:20.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:19 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:20.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:19 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:20.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:19 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:20.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:19 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:20.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:19 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:20.694 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:10:20 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:10:20.385+0000 7fb4d5070640 -1 osd.9 0 log_to_monitors true 2024-09-11T22:10:20.946 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:20 smithi092 ceph-mon[31173]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:20.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:20 smithi092 ceph-mon[31173]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:10:21.107 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:20 smithi110 ceph-mon[34830]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:21.107 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:20 smithi110 ceph-mon[34830]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:10:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:20 smithi153 ceph-mon[35045]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:20 smithi153 ceph-mon[35045]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:10:22.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:21 smithi092 ceph-mon[31173]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:10:22.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:21 smithi092 ceph-mon[31173]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T22:10:22.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:21 smithi092 ceph-mon[31173]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:10:22.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:22.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:21 smithi092 ceph-mon[31173]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:10:22.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:21 smithi092 ceph-mon[31173]: osdmap e65: 10 total, 9 up, 10 in 2024-09-11T22:10:22.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:21 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:22.189 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:21 smithi110 ceph-mon[34830]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:10:22.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:21 smithi110 ceph-mon[34830]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T22:10:22.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:21 smithi110 ceph-mon[34830]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:10:22.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:22.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:21 smithi110 ceph-mon[34830]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:10:22.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:21 smithi110 ceph-mon[34830]: osdmap e65: 10 total, 9 up, 10 in 2024-09-11T22:10:22.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:21 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:21 smithi153 ceph-mon[35045]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:10:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:21 smithi153 ceph-mon[35045]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T22:10:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:21 smithi153 ceph-mon[35045]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T22:10:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:21 smithi153 ceph-mon[35045]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T22:10:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:21 smithi153 ceph-mon[35045]: osdmap e65: 10 total, 9 up, 10 in 2024-09-11T22:10:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:21 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:22.470 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:10:22.206+0000 7fb4d1807640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T22:10:22.471 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:10:22.217+0000 7fb4c8e13640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:10:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T22:10:22.512 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2024-09-11T22:10:22.774 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:10:23.056 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:23.057 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.058 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:23.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 -- 172.21.15.110:0/2648619050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa694101d00 msgr2=0x7fa694102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:23.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 --2- 172.21.15.110:0/2648619050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa694101d00 0x7fa694102180 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa684009900 tx=0x7fa68402a410 comp rx=0 tx=0).stop 2024-09-11T22:10:23.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 -- 172.21.15.110:0/2648619050 shutdown_connections 2024-09-11T22:10:23.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 --2- 172.21.15.110:0/2648619050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6941026c0 0x7fa694108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 --2- 172.21.15.110:0/2648619050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa694101d00 0x7fa694102180 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 --2- 172.21.15.110:0/2648619050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa694100b00 0x7fa694100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 -- 172.21.15.110:0/2648619050 >> 172.21.15.110:0/2648619050 conn(0x7fa6940fc290 msgr2=0x7fa6940fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:10:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 -- 172.21.15.110:0/2648619050 shutdown_connections 2024-09-11T22:10:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.246+0000 7fa698daa640 1 -- 172.21.15.110:0/2648619050 wait complete. 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.247+0000 7fa698daa640 1 Processor -- start 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.247+0000 7fa698daa640 1 -- start start 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.247+0000 7fa698daa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694100b00 0x7fa694196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.247+0000 7fa698daa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa694101d00 0x7fa694197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa698daa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6941026c0 0x7fa69419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa698daa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410c610 con 0x7fa694101d00 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa698daa640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410c490 con 0x7fa6941026c0 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa698daa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410c790 con 0x7fa694100b00 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6941026c0 0x7fa69419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6941026c0 0x7fa69419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57938/0 (socket says 172.21.15.110:57938) 2024-09-11T22:10:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 -- 172.21.15.110:0/2867952127 learned_addr learned my addr 172.21.15.110:0/2867952127 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa69259c640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694100b00 0x7fa694196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa691d9b640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa694101d00 0x7fa694197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 -- 172.21.15.110:0/2867952127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694100b00 msgr2=0x7fa694196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694100b00 0x7fa694196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 -- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa694101d00 msgr2=0x7fa694197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa694101d00 0x7fa694197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 -- 172.21.15.110:0/2867952127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69419e900 con 0x7fa6941026c0 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa691d9b640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa694101d00 0x7fa694197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa692d9d640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6941026c0 0x7fa69419e260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00cce0 tx=0x7fa68c00a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa6837fe640 1 -- 172.21.15.110:0/2867952127 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00f450 con 0x7fa6941026c0 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa6837fe640 1 -- 172.21.15.110:0/2867952127 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa68c00faa0 con 0x7fa6941026c0 2024-09-11T22:10:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69419ebf0 con 0x7fa6941026c0 2024-09-11T22:10:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.248+0000 7fa6837fe640 1 -- 172.21.15.110:0/2867952127 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c018b70 con 0x7fa6941026c0 2024-09-11T22:10:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.249+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69419f0e0 con 0x7fa6941026c0 2024-09-11T22:10:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.250+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa694107fa0 con 0x7fa6941026c0 2024-09-11T22:10:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.251+0000 7fa6837fe640 1 -- 172.21.15.110:0/2867952127 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa68c018d10 con 0x7fa6941026c0 2024-09-11T22:10:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.252+0000 7fa6837fe640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa67c0a2b80 0x7fa67c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.252+0000 7fa6837fe640 1 -- 172.21.15.110:0/2867952127 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6261+0+0 (secure 0 0 0) 0x7fa68c0cec10 con 0x7fa6941026c0 2024-09-11T22:10:23.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.252+0000 7fa69259c640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa67c0a2b80 0x7fa67c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:23.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.253+0000 7fa69259c640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa67c0a2b80 0x7fa67c0a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6941981c0 tx=0x7fa68800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:10:23.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.256+0000 7fa6837fe640 1 -- 172.21.15.110:0/2867952127 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa68c014030 con 0x7fa6941026c0 2024-09-11T22:10:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439]' entity='osd.9' 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:23.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:23.438+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa694062630 con 0x7fa67c0a2b80 2024-09-11T22:10:24.089 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi092 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] boot 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:10:24.090 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:24.170 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi092 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] boot 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:10:24.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi092 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: osd.9 [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] boot 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:10:24.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:10:24.191 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:10:24.191 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:25.079 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:24 smithi110 ceph-mon[34830]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:25.079 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:24 smithi110 ceph-mon[34830]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:25.079 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:24 smithi110 ceph-mon[34830]: osdmap e67: 10 total, 10 up, 10 in 2024-09-11T22:10:25.190 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:24 smithi092 ceph-mon[31173]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:25.190 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:24 smithi092 ceph-mon[31173]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:25.190 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:24 smithi092 ceph-mon[31173]: osdmap e67: 10 total, 10 up, 10 in 2024-09-11T22:10:25.206 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:24 smithi153 ceph-mon[35045]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:25.207 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:24 smithi153 ceph-mon[35045]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:10:25.207 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:24 smithi153 ceph-mon[35045]: osdmap e67: 10 total, 10 up, 10 in 2024-09-11T22:10:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:26 smithi110 ceph-mon[34830]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:26 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/2844257577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]: dispatch 2024-09-11T22:10:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:26 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]: dispatch 2024-09-11T22:10:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:26 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]': finished 2024-09-11T22:10:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:26 smithi110 ceph-mon[34830]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T22:10:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:26 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:27.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:26 smithi110 ceph-mon[34830]: from='client.? 172.21.15.110:0/962859101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:27.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:26 smithi092 ceph-mon[31173]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:26 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/2844257577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]: dispatch 2024-09-11T22:10:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:26 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]: dispatch 2024-09-11T22:10:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:26 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]': finished 2024-09-11T22:10:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:26 smithi092 ceph-mon[31173]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T22:10:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:26 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:26 smithi092 ceph-mon[31173]: from='client.? 172.21.15.110:0/962859101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:26 smithi153 ceph-mon[35045]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:27.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:26 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/2844257577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]: dispatch 2024-09-11T22:10:27.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:26 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]: dispatch 2024-09-11T22:10:27.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:26 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abd86e0e-ad63-46dd-961a-54b939ad22b9"}]': finished 2024-09-11T22:10:27.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:26 smithi153 ceph-mon[35045]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T22:10:27.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:26 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:27.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:26 smithi153 ceph-mon[35045]: from='client.? 172.21.15.110:0/962859101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:29.143 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:28 smithi110 ceph-mon[34830]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:29.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:28 smithi092 ceph-mon[31173]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:29.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:28 smithi153 ceph-mon[35045]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:30.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:30 smithi110 ceph-mon[34830]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:30 smithi092 ceph-mon[31173]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:31.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:30 smithi153 ceph-mon[35045]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:32.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:31 smithi110 ceph-mon[34830]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:32.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:31 smithi092 ceph-mon[31173]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:32.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:31 smithi153 ceph-mon[35045]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:34 smithi110 ceph-mon[34830]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:10:34.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:34 smithi092 ceph-mon[31173]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:10:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:35.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:34 smithi153 ceph-mon[35045]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:35.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:10:35.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:35.953 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:35 smithi110 ceph-mon[34830]: Deploying daemon osd.10 on smithi110 2024-09-11T22:10:36.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:35 smithi092 ceph-mon[31173]: Deploying daemon osd.10 on smithi110 2024-09-11T22:10:36.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:35 smithi153 ceph-mon[35045]: Deploying daemon osd.10 on smithi110 2024-09-11T22:10:36.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:36 smithi153 ceph-mon[35045]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:36.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:36 smithi110 ceph-mon[34830]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:37.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:36 smithi092 ceph-mon[31173]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:39.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:38 smithi092 ceph-mon[31173]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:39.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:38 smithi153 ceph-mon[35045]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:39.116 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:38 smithi110 ceph-mon[34830]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:41.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:40 smithi092 ceph-mon[31173]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:41.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:41.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:41.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:41.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:40 smithi153 ceph-mon[35045]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:41.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:41.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:41.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:41.054 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:40 smithi110 ceph-mon[34830]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:41.054 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:41.054 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:41.054 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:42.900 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:42 smithi110 ceph-mon[34830]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:42 smithi092 ceph-mon[31173]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:43.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:42 smithi153 ceph-mon[35045]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:43.897 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 10 on host 'smithi110' 2024-09-11T22:10:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.896+0000 7fa6837fe640 1 -- 172.21.15.110:0/2867952127 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa694062630 con 0x7fa67c0a2b80 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa67c0a2b80 msgr2=0x7fa67c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa67c0a2b80 0x7fa67c0a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6941981c0 tx=0x7fa68800b410 comp rx=0 tx=0).stop 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6941026c0 msgr2=0x7fa69419e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6941026c0 0x7fa69419e260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00cce0 tx=0x7fa68c00a620 comp rx=0 tx=0).stop 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 shutdown_connections 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa67c0a2b80 0x7fa67c0a5040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6941026c0 0x7fa69419e260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa694101d00 0x7fa694197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 --2- 172.21.15.110:0/2867952127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694100b00 0x7fa694196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 >> 172.21.15.110:0/2867952127 conn(0x7fa6940fc290 msgr2=0x7fa694105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:10:43.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 shutdown_connections 2024-09-11T22:10:43.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T22:10:43.899+0000 7fa698daa640 1 -- 172.21.15.110:0/2867952127 wait complete. 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:44.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:44.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:44.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:44.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:44.637 DEBUG:teuthology.orchestra.run.smithi110:osd.10> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T22:10:44.640 INFO:tasks.cephadm:Deploying osd.11 on smithi153 with /dev/vg_nvme/lv_1... 2024-09-11T22:10:44.641 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:10:44.895 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:10:46.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:46 smithi153 ceph-mon[35045]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:46.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:46 smithi153 ceph-mon[35045]: from='osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:10:46.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:46 smithi153 ceph-mon[35045]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:10:46.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:46 smithi110 ceph-mon[34830]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:46.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:46 smithi110 ceph-mon[34830]: from='osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:10:46.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:46 smithi110 ceph-mon[34830]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:10:46.872 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:10:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:10:46.478+0000 7fdc2fec2640 -1 osd.10 0 log_to_monitors true 2024-09-11T22:10:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:46 smithi092 ceph-mon[31173]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:46 smithi092 ceph-mon[31173]: from='osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:10:47.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:46 smithi092 ceph-mon[31173]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:10:47.758 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-11T22:10:47.784 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_1 2024-09-11T22:10:48.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:10:48.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:48.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:10:48.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: osdmap e70: 11 total, 10 up, 11 in 2024-09-11T22:10:48.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:47 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:48.038 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:48.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:10:48.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: osdmap e70: 11 total, 10 up, 11 in 2024-09-11T22:10:48.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:47 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:48.217 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:10:47.725+0000 7fdc2be45640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T22:10:48.217 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:10:47.742+0000 7fdc27c6d640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:10:48.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:48.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:10:48.219 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:10:48.219 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T22:10:48.219 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: osdmap e70: 11 total, 10 up, 11 in 2024-09-11T22:10:48.219 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:47 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:48.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 -- 172.21.15.153:0/448448258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168102700 msgr2=0x7f8168104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:48.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 --2- 172.21.15.153:0/448448258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168102700 0x7f8168104af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8158009900 tx=0x7f815802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:10:48.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 -- 172.21.15.153:0/448448258 shutdown_connections 2024-09-11T22:10:48.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 --2- 172.21.15.153:0/448448258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8168107960 0x7f8168109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:48.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 --2- 172.21.15.153:0/448448258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8168105030 0x7f8168107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:48.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 --2- 172.21.15.153:0/448448258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168102700 0x7f8168104af0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:48.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 -- 172.21.15.153:0/448448258 >> 172.21.15.153:0/448448258 conn(0x7f81680fc280 msgr2=0x7f81680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 -- 172.21.15.153:0/448448258 shutdown_connections 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 -- 172.21.15.153:0/448448258 wait complete. 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.484+0000 7f816cae7640 1 Processor -- start 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816cae7640 1 -- start start 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816cae7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8168102700 0x7f8168197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816cae7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168105030 0x7f81681983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816cae7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8168107960 0x7f816819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816cae7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f816810d780 con 0x7f8168107960 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816cae7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f816810d600 con 0x7f8168102700 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816cae7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f816810d900 con 0x7f8168105030 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168105030 0x7f81681983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168105030 0x7f81681983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49766/0 (socket says 172.21.15.153:49766) 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 -- 172.21.15.153:0/2596924791 learned_addr learned my addr 172.21.15.153:0/2596924791 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f816659c640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8168102700 0x7f8168197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 -- 172.21.15.153:0/2596924791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8168102700 msgr2=0x7f8168197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8168102700 0x7f8168197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8166d9d640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8168107960 0x7f816819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 -- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8168107960 msgr2=0x7f816819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8168107960 0x7f816819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 -- 172.21.15.153:0/2596924791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816819faa0 con 0x7f8168105030 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8166d9d640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8168107960 0x7f816819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f8165d9b640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168105030 0x7f81681983b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b450 tx=0x7f815c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.485+0000 7f81577fe640 1 -- 172.21.15.153:0/2596924791 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c015020 con 0x7f8168105030 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.486+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816819fd90 con 0x7f8168105030 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.486+0000 7f81577fe640 1 -- 172.21.15.153:0/2596924791 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f815c011d40 con 0x7f8168105030 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.486+0000 7f81577fe640 1 -- 172.21.15.153:0/2596924791 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c00f340 con 0x7f8168105030 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.486+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a02d0 con 0x7f8168105030 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.487+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168105410 con 0x7f8168105030 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.488+0000 7f81577fe640 1 -- 172.21.15.153:0/2596924791 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f815c018020 con 0x7f8168105030 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.488+0000 7f81577fe640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81500a2bb0 0x7f81500a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:10:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.488+0000 7f81577fe640 1 -- 172.21.15.153:0/2596924791 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6414+0+0 (secure 0 0 0) 0x7f815c0ce3c0 con 0x7f8168105030 2024-09-11T22:10:48.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.489+0000 7f816659c640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81500a2bb0 0x7f81500a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:10:48.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.489+0000 7f816659c640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81500a2bb0 0x7f81500a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8158000c00 tx=0x7f81580314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:10:48.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.493+0000 7f81577fe640 1 -- 172.21.15.153:0/2596924791 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f815c01f020 con 0x7f8168105030 2024-09-11T22:10:48.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:10:48.668+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8168062630 con 0x7f81500a2bb0 2024-09-11T22:10:49.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:48 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi110 2024-09-11T22:10:49.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:48 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-11T22:10:49.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:48 smithi092 ceph-mon[31173]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:49.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:49.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:10:49.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:10:49.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:48 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi110 2024-09-11T22:10:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:48 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-11T22:10:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:48 smithi153 ceph-mon[35045]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:10:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:10:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:48 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi110 2024-09-11T22:10:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:48 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-11T22:10:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:48 smithi110 ceph-mon[34830]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:10:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:10:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:10:50.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:49 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:10:50.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:49 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:10:50.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:49 smithi092 ceph-mon[31173]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:50.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:49 smithi092 ceph-mon[31173]: osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] boot 2024-09-11T22:10:50.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:49 smithi092 ceph-mon[31173]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T22:10:50.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:49 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:50.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:49 smithi092 ceph-mon[31173]: osdmap e72: 11 total, 11 up, 11 in 2024-09-11T22:10:50.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:49 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:10:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:49 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:10:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:49 smithi153 ceph-mon[35045]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:49 smithi153 ceph-mon[35045]: osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] boot 2024-09-11T22:10:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:49 smithi153 ceph-mon[35045]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T22:10:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:49 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:49 smithi153 ceph-mon[35045]: osdmap e72: 11 total, 11 up, 11 in 2024-09-11T22:10:50.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:49 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:10:50.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:49 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:10:50.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:49 smithi110 ceph-mon[34830]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:10:50.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:49 smithi110 ceph-mon[34830]: osd.10 [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] boot 2024-09-11T22:10:50.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:49 smithi110 ceph-mon[34830]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T22:10:50.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:49 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:10:50.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:49 smithi110 ceph-mon[34830]: osdmap e72: 11 total, 11 up, 11 in 2024-09-11T22:10:50.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:50 smithi153 ceph-mon[35045]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:50.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:50 smithi153 ceph-mon[35045]: osdmap e73: 11 total, 11 up, 11 in 2024-09-11T22:10:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:50 smithi092 ceph-mon[31173]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:50 smithi092 ceph-mon[31173]: osdmap e73: 11 total, 11 up, 11 in 2024-09-11T22:10:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:50 smithi110 ceph-mon[34830]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:10:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:50 smithi110 ceph-mon[34830]: osdmap e73: 11 total, 11 up, 11 in 2024-09-11T22:10:52.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: osdmap e74: 11 total, 11 up, 11 in 2024-09-11T22:10:52.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/2694379341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]: dispatch 2024-09-11T22:10:52.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]: dispatch 2024-09-11T22:10:52.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]': finished 2024-09-11T22:10:52.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T22:10:52.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:10:52.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:10:52.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:52 smithi110 ceph-mon[34830]: from='client.? 172.21.15.153:0/2963679792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:52.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: osdmap e74: 11 total, 11 up, 11 in 2024-09-11T22:10:52.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/2694379341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]: dispatch 2024-09-11T22:10:52.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]: dispatch 2024-09-11T22:10:52.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]': finished 2024-09-11T22:10:52.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T22:10:52.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:10:52.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:10:52.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:52 smithi092 ceph-mon[31173]: from='client.? 172.21.15.153:0/2963679792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: osdmap e74: 11 total, 11 up, 11 in 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/2694379341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]: dispatch 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]: dispatch 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74de56f5-f0f9-44d6-ac7a-c7edacd28a4a"}]': finished 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:10:52.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:52 smithi153 ceph-mon[35045]: from='client.? 172.21.15.153:0/2963679792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:10:55.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:54 smithi092 ceph-mon[31173]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:10:55.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:54 smithi153 ceph-mon[35045]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:10:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:54 smithi110 ceph-mon[34830]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:10:57.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:56 smithi092 ceph-mon[31173]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T22:10:57.019 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:56 smithi110 ceph-mon[34830]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T22:10:57.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:56 smithi153 ceph-mon[35045]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T22:10:59.000 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:10:58 smithi153 ceph-mon[35045]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T22:10:59.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:10:58 smithi092 ceph-mon[31173]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T22:10:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:10:58 smithi110 ceph-mon[34830]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T22:11:00.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:00 smithi153 ceph-mon[35045]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T22:11:00.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:11:00.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:00 smithi092 ceph-mon[31173]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T22:11:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:11:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:00 smithi110 ceph-mon[34830]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T22:11:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:11:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:01.951 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:01 smithi153 ceph-mon[35045]: Deploying daemon osd.11 on smithi153 2024-09-11T22:11:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:01 smithi092 ceph-mon[31173]: Deploying daemon osd.11 on smithi153 2024-09-11T22:11:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:01 smithi110 ceph-mon[34830]: Deploying daemon osd.11 on smithi153 2024-09-11T22:11:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:02 smithi092 ceph-mon[31173]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T22:11:03.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:02 smithi153 ceph-mon[35045]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T22:11:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:02 smithi110 ceph-mon[34830]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T22:11:04.983 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:04 smithi153 ceph-mon[35045]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:11:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:04 smithi092 ceph-mon[31173]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:11:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:04 smithi110 ceph-mon[34830]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:11:07.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:06 smithi092 ceph-mon[31173]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:07.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:07.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:07.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:07.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:06 smithi153 ceph-mon[35045]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:07.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:06 smithi110 ceph-mon[34830]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:08 smithi092 ceph-mon[31173]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:08 smithi153 ceph-mon[35045]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:08 smithi110 ceph-mon[34830]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:10.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.275+0000 7f81577fe640 1 -- 172.21.15.153:0/2596924791 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8168062630 con 0x7f81500a2bb0 2024-09-11T22:11:10.276 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 11 on host 'smithi153' 2024-09-11T22:11:10.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81500a2bb0 msgr2=0x7f81500a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:10.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81500a2bb0 0x7f81500a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8158000c00 tx=0x7f81580314a0 comp rx=0 tx=0).stop 2024-09-11T22:11:10.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168105030 msgr2=0x7f81681983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:10.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168105030 0x7f81681983b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b450 tx=0x7f815c00b910 comp rx=0 tx=0).stop 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 shutdown_connections 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f81500a2bb0 0x7f81500a5070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8168107960 0x7f816819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8168105030 0x7f81681983b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 --2- 172.21.15.153:0/2596924791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8168102700 0x7f8168197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 >> 172.21.15.153:0/2596924791 conn(0x7f81680fc280 msgr2=0x7f81681030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 shutdown_connections 2024-09-11T22:11:10.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:10.278+0000 7f816cae7640 1 -- 172.21.15.153:0/2596924791 wait complete. 2024-09-11T22:11:10.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:10 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:10 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:10.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:10.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:10 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:11.017 DEBUG:teuthology.orchestra.run.smithi153:osd.11> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T22:11:11.019 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T22:11:11.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:11:11.277 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 -- 172.21.15.92:0/1088552401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 msgr2=0x7f58b4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 --2- 172.21.15.92:0/1088552401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4100ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009900 tx=0x7f58a802a410 comp rx=0 tx=0).stop 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 -- 172.21.15.92:0/1088552401 shutdown_connections 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 --2- 172.21.15.92:0/1088552401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b4102690 0x7f58b4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 --2- 172.21.15.92:0/1088552401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4101cd0 0x7f58b4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 --2- 172.21.15.92:0/1088552401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4100ed0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 -- 172.21.15.92:0/1088552401 >> 172.21.15.92:0/1088552401 conn(0x7f58b40fc280 msgr2=0x7f58b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 -- 172.21.15.92:0/1088552401 shutdown_connections 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 -- 172.21.15.92:0/1088552401 wait complete. 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.734+0000 7f58b88a1640 1 Processor -- start 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b88a1640 1 -- start start 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b88a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b88a1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4101cd0 0x7f58b41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b88a1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b4102690 0x7f58b41035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b88a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410c5e0 con 0x7f58b4100ad0 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b88a1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410c460 con 0x7f58b4102690 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b88a1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410c760 con 0x7f58b4101cd0 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47660/0 (socket says 172.21.15.92:47660) 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 -- 172.21.15.92:0/3714681073 learned_addr learned my addr 172.21.15.92:0/3714681073 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b1d9b640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4101cd0 0x7f58b41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b2d9d640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b4102690 0x7f58b41035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 -- 172.21.15.92:0/3714681073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4101cd0 msgr2=0x7f58b41030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4101cd0 0x7f58b41030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 -- 172.21.15.92:0/3714681073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b4102690 msgr2=0x7f58b41035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b4102690 0x7f58b41035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 -- 172.21.15.92:0/3714681073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b4103eb0 con 0x7f58b4100ad0 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b1d9b640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4101cd0 0x7f58b41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.735+0000 7f58b259c640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4104a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58a80314e0 tx=0x7f58a80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.736+0000 7f58a37fe640 1 -- 172.21.15.92:0/3714681073 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a803f030 con 0x7f58b4100ad0 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.736+0000 7f58a37fe640 1 -- 172.21.15.92:0/3714681073 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58a8003cb0 con 0x7f58b4100ad0 2024-09-11T22:11:11.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.736+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b41a1fa0 con 0x7f58b4100ad0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.736+0000 7f58a37fe640 1 -- 172.21.15.92:0/3714681073 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8035070 con 0x7f58b4100ad0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.736+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b41a2340 con 0x7f58b4100ad0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.737+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b41a2630 con 0x7f58b4100ad0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.738+0000 7f58a37fe640 1 -- 172.21.15.92:0/3714681073 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f58a8042030 con 0x7f58b4100ad0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.739+0000 7f58a37fe640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f589c0a2b80 0x7f589c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.739+0000 7f58a37fe640 1 -- 172.21.15.92:0/3714681073 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6809+0+0 (secure 0 0 0) 0x7f58a80eefa0 con 0x7f58b4100ad0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.739+0000 7f58b1d9b640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f589c0a2b80 0x7f589c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.739+0000 7f58b1d9b640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f589c0a2b80 0x7f589c0a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58b41047e0 tx=0x7f58a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:11.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.743+0000 7f58a37fe640 1 -- 172.21.15.92:0/3714681073 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58a80bbe90 con 0x7f58b4100ad0 2024-09-11T22:11:11.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.916+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f58b41a2920 con 0x7f58b4100ad0 2024-09-11T22:11:11.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:11 smithi092 ceph-mon[31173]: from='osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:11:11.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:11 smithi092 ceph-mon[31173]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:11:11.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.917+0000 7f58a37fe640 1 -- 172.21.15.92:0/3714681073 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f58a80c0190 con 0x7f58b4100ad0 2024-09-11T22:11:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:11.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f589c0a2b80 msgr2=0x7f589c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:11.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f589c0a2b80 0x7f589c0a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58b41047e0 tx=0x7f58a400b390 comp rx=0 tx=0).stop 2024-09-11T22:11:11.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 msgr2=0x7f58b4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:11.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4104a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58a80314e0 tx=0x7f58a80042e0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 shutdown_connections 2024-09-11T22:11:11.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f589c0a2b80 0x7f589c0a5040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b4102690 0x7f58b41035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4101cd0 0x7f58b41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 --2- 172.21.15.92:0/3714681073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b4100ad0 0x7f58b4104a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:11.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 >> 172.21.15.92:0/3714681073 conn(0x7f58b40fc280 msgr2=0x7f58b4069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:11.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 shutdown_connections 2024-09-11T22:11:11.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:11.919+0000 7f58b88a1640 1 -- 172.21.15.92:0/3714681073 wait complete. 2024-09-11T22:11:12.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:11 smithi153 ceph-mon[35045]: from='osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:11:12.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:11 smithi153 ceph-mon[35045]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:11:12.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:11:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:11:11.704+0000 7f0768a3e640 -1 osd.11 0 log_to_monitors true 2024-09-11T22:11:12.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:11 smithi110 ceph-mon[34830]: from='osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:11:12.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:11 smithi110 ceph-mon[34830]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:11:12.812 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1726092648,"num_in_osds":12,"osd_in_since":1726092651,"num_remapped_pgs":0} 2024-09-11T22:11:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-mon[35045]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-mon[35045]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:11:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-mon[35045]: from='osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:11:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-mon[35045]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T22:11:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-mon[35045]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:11:13.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3714681073' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:11:13.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:11:12.773+0000 7f07649c1640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T22:11:13.040 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:11:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:11:12.789+0000 7f07607e9640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:11:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:12 smithi110 ceph-mon[34830]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:12 smithi110 ceph-mon[34830]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:11:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:12 smithi110 ceph-mon[34830]: from='osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:11:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:12 smithi110 ceph-mon[34830]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T22:11:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:12 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:12 smithi110 ceph-mon[34830]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:11:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:12 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3714681073' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:11:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:12 smithi092 ceph-mon[31173]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:13.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:12 smithi092 ceph-mon[31173]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:11:13.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:12 smithi092 ceph-mon[31173]: from='osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:11:13.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:12 smithi092 ceph-mon[31173]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T22:11:13.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:12 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:13.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:12 smithi092 ceph-mon[31173]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-11T22:11:13.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:12 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3714681073' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:11:13.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:11:14.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:13 smithi153 ceph-mon[35045]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:11:14.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:13 smithi153 ceph-mon[35045]: osdmap e77: 12 total, 11 up, 12 in 2024-09-11T22:11:14.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:13 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:14.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:13 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:14.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:13 smithi092 ceph-mon[31173]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:11:14.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:13 smithi092 ceph-mon[31173]: osdmap e77: 12 total, 11 up, 12 in 2024-09-11T22:11:14.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:13 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:14.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:13 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:13 smithi110 ceph-mon[34830]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-11T22:11:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:13 smithi110 ceph-mon[34830]: osdmap e77: 12 total, 11 up, 12 in 2024-09-11T22:11:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:13 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:13 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 -- 172.21.15.92:0/2475158578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c101d10 msgr2=0x7fa58c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 --2- 172.21.15.92:0/2475158578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c101d10 0x7fa58c102190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa580009900 tx=0x7fa58002a410 comp rx=0 tx=0).stop 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 -- 172.21.15.92:0/2475158578 shutdown_connections 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 --2- 172.21.15.92:0/2475158578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa58c1026d0 0x7fa58c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 --2- 172.21.15.92:0/2475158578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c101d10 0x7fa58c102190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 --2- 172.21.15.92:0/2475158578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa58c100b10 0x7fa58c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 -- 172.21.15.92:0/2475158578 >> 172.21.15.92:0/2475158578 conn(0x7fa58c0fc2a0 msgr2=0x7fa58c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 -- 172.21.15.92:0/2475158578 shutdown_connections 2024-09-11T22:11:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.516+0000 7fa590c44640 1 -- 172.21.15.92:0/2475158578 wait complete. 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 Processor -- start 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 -- start start 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa58c100b10 0x7fa58c196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa58c101d10 0x7fa58c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c1026d0 0x7fa58c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c10c620 con 0x7fa58c1026d0 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c10c4a0 con 0x7fa58c101d10 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.517+0000 7fa590c44640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c10c7a0 con 0x7fa58c100b10 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c1026d0 0x7fa58c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c1026d0 0x7fa58c19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47672/0 (socket says 172.21.15.92:47672) 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 -- 172.21.15.92:0/3225159107 learned_addr learned my addr 172.21.15.92:0/3225159107 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58a59c640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa58c100b10 0x7fa58c196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa589d9b640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa58c101d10 0x7fa58c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 -- 172.21.15.92:0/3225159107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa58c100b10 msgr2=0x7fa58c196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa58c100b10 0x7fa58c196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 -- 172.21.15.92:0/3225159107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa58c101d10 msgr2=0x7fa58c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa58c101d10 0x7fa58c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 -- 172.21.15.92:0/3225159107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c19e990 con 0x7fa58c1026d0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa58ad9d640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c1026d0 0x7fa58c19e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa58400cad0 tx=0x7fa58400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa57b7fe640 1 -- 172.21.15.92:0/3225159107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58400ae20 con 0x7fa58c1026d0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c19ec80 con 0x7fa58c1026d0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa57b7fe640 1 -- 172.21.15.92:0/3225159107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa584005020 con 0x7fa58c1026d0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa57b7fe640 1 -- 172.21.15.92:0/3225159107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58400f6b0 con 0x7fa58c1026d0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.518+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c06a790 con 0x7fa58c1026d0 2024-09-11T22:11:14.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.519+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58c06b210 con 0x7fa58c1026d0 2024-09-11T22:11:14.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.520+0000 7fa57b7fe640 1 -- 172.21.15.92:0/3225159107 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5840051d0 con 0x7fa58c1026d0 2024-09-11T22:11:14.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.521+0000 7fa57b7fe640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5740a4dc0 0x7fa5740a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:14.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.521+0000 7fa58a59c640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5740a4dc0 0x7fa5740a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:14.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.521+0000 7fa57b7fe640 1 -- 172.21.15.92:0/3225159107 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fa5840cec10 con 0x7fa58c1026d0 2024-09-11T22:11:14.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.521+0000 7fa58a59c640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5740a4dc0 0x7fa5740a7280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa58c101b70 tx=0x7fa57c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:14.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.525+0000 7fa57b7fe640 1 -- 172.21.15.92:0/3225159107 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa584014030 con 0x7fa58c1026d0 2024-09-11T22:11:14.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.692+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa58c107ab0 con 0x7fa58c1026d0 2024-09-11T22:11:14.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.692+0000 7fa57b7fe640 1 -- 172.21.15.92:0/3225159107 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7fa58409c010 con 0x7fa58c1026d0 2024-09-11T22:11:14.694 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5740a4dc0 msgr2=0x7fa5740a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5740a4dc0 0x7fa5740a7280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa58c101b70 tx=0x7fa57c00a2d0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c1026d0 msgr2=0x7fa58c19e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c1026d0 0x7fa58c19e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa58400cad0 tx=0x7fa58400cf90 comp rx=0 tx=0).stop 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 shutdown_connections 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5740a4dc0 0x7fa5740a7280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa58c1026d0 0x7fa58c19e320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa58c101d10 0x7fa58c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 --2- 172.21.15.92:0/3225159107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa58c100b10 0x7fa58c196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:14.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 >> 172.21.15.92:0/3225159107 conn(0x7fa58c0fc2a0 msgr2=0x7fa58c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:14.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 shutdown_connections 2024-09-11T22:11:14.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:14.695+0000 7fa590c44640 1 -- 172.21.15.92:0/3225159107 wait complete. 2024-09-11T22:11:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: purged_snaps scrub starts 2024-09-11T22:11:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: purged_snaps scrub ok 2024-09-11T22:11:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] boot 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:14 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3225159107' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: purged_snaps scrub starts 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: purged_snaps scrub ok 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] boot 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:14 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3225159107' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:11:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: purged_snaps scrub starts 2024-09-11T22:11:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: purged_snaps scrub ok 2024-09-11T22:11:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:11:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: osd.11 [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] boot 2024-09-11T22:11:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:11:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:11:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:15.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:14 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3225159107' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:11:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1726092673,"num_in_osds":12,"osd_in_since":1726092651,"num_remapped_pgs":0} 2024-09-11T22:11:15.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:16.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:15 smithi092 ceph-mon[31173]: Detected new or changed devices on smithi153 2024-09-11T22:11:16.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:15 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-11T22:11:16.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:15 smithi092 ceph-mon[31173]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:11:16.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:15 smithi153 ceph-mon[35045]: Detected new or changed devices on smithi153 2024-09-11T22:11:16.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:15 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-11T22:11:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:15 smithi153 ceph-mon[35045]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 -- 172.21.15.92:0/3069274502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 msgr2=0x7f9bb41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/3069274502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41067e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009900 tx=0x7f9ba802a410 comp rx=0 tx=0).stop 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 -- 172.21.15.92:0/3069274502 shutdown_connections 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/3069274502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb4106f10 0x7f9bb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/3069274502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41067e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/3069274502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bb4101de0 0x7f9bb41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 -- 172.21.15.92:0/3069274502 >> 172.21.15.92:0/3069274502 conn(0x7f9bb40fd8c0 msgr2=0x7f9bb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 -- 172.21.15.92:0/3069274502 shutdown_connections 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.024+0000 7f9bb98e6640 1 -- 172.21.15.92:0/3069274502 wait complete. 2024-09-11T22:11:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 Processor -- start 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 -- start start 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb4101de0 0x7f9bb4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bb4106f10 0x7f9bb419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4111e60 con 0x7f9bb4102790 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4111ce0 con 0x7f9bb4101de0 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb98e6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4111fe0 con 0x7f9bb4106f10 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40056/0 (socket says 172.21.15.92:40056) 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb27fc640 1 -- 172.21.15.92:0/641330736 learned_addr learned my addr 172.21.15.92:0/641330736 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb2ffd640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb4101de0 0x7f9bb4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb27fc640 1 -- 172.21.15.92:0/641330736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bb4106f10 msgr2=0x7f9bb419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.025+0000 7f9bb37fe640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bb4106f10 0x7f9bb419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb27fc640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bb4106f10 0x7f9bb419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb27fc640 1 -- 172.21.15.92:0/641330736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb4101de0 msgr2=0x7f9bb4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb27fc640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb4101de0 0x7f9bb4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb27fc640 1 -- 172.21.15.92:0/641330736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb419d430 con 0x7f9bb4102790 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb27fc640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41989c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8006010 tx=0x7f9ba8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb88e4640 1 -- 172.21.15.92:0/641330736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba803f030 con 0x7f9bb4102790 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb88e4640 1 -- 172.21.15.92:0/641330736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ba80035f0 con 0x7f9bb4102790 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb419d6c0 con 0x7f9bb4102790 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb88e4640 1 -- 172.21.15.92:0/641330736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8035070 con 0x7f9bb4102790 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.026+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb41a5000 con 0x7f9bb4102790 2024-09-11T22:11:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.027+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb41032c0 con 0x7f9bb4102790 2024-09-11T22:11:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.028+0000 7f9bb88e4640 1 -- 172.21.15.92:0/641330736 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ba8003b10 con 0x7f9bb4102790 2024-09-11T22:11:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.029+0000 7f9bb88e4640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9b9c0a2b80 0x7f9b9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.029+0000 7f9bb2ffd640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9b9c0a2b80 0x7f9b9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.029+0000 7f9bb88e4640 1 -- 172.21.15.92:0/641330736 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f9ba80ef720 con 0x7f9bb4102790 2024-09-11T22:11:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.029+0000 7f9bb2ffd640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9b9c0a2b80 0x7f9b9c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40058c0 tx=0x7f9ba400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.033+0000 7f9bb88e4640 1 -- 172.21.15.92:0/641330736 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ba80bcb20 con 0x7f9bb4102790 2024-09-11T22:11:16.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.200+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bb410a0f0 con 0x7f9bb4102790 2024-09-11T22:11:16.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.201+0000 7f9bb88e4640 1 -- 172.21.15.92:0/641330736 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18467 (secure 0 0 0) 0x7f9ba80bc420 con 0x7f9bb4102790 2024-09-11T22:11:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:16.203 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":79,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:11:15.212849+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:11:16.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9b9c0a2b80 msgr2=0x7f9b9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9b9c0a2b80 0x7f9b9c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40058c0 tx=0x7f9ba400a250 comp rx=0 tx=0).stop 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 msgr2=0x7f9bb41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41989c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8006010 tx=0x7f9ba8004270 comp rx=0 tx=0).stop 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 shutdown_connections 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9b9c0a2b80 0x7f9b9c0a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bb4106f10 0x7f9bb419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb4102790 0x7f9bb41989c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 --2- 172.21.15.92:0/641330736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb4101de0 0x7f9bb4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.203+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 >> 172.21.15.92:0/641330736 conn(0x7f9bb40fd8c0 msgr2=0x7f9bb40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.204+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 shutdown_connections 2024-09-11T22:11:16.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:16.204+0000 7f9bb98e6640 1 -- 172.21.15.92:0/641330736 wait complete. 2024-09-11T22:11:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:15 smithi110 ceph-mon[34830]: Detected new or changed devices on smithi153 2024-09-11T22:11:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:15 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-11T22:11:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:15 smithi110 ceph-mon[34830]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:11:16.830 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:07:29.899850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T22:11:16.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T22:11:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:16 smithi153 ceph-mon[35045]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:16 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/641330736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:11:17.071 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:17.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:16 smithi092 ceph-mon[31173]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:17.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:16 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/641330736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:11:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:16 smithi110 ceph-mon[34830]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:16 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/641330736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 -- 172.21.15.92:0/1020993742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100cc0 msgr2=0x7f4274106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 --2- 172.21.15.92:0/1020993742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100cc0 0x7f4274106800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f426802a410 comp rx=0 tx=0).stop 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 -- 172.21.15.92:0/1020993742 shutdown_connections 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 --2- 172.21.15.92:0/1020993742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274106f30 0x7f427410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 --2- 172.21.15.92:0/1020993742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100cc0 0x7f4274106800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 --2- 172.21.15.92:0/1020993742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4274100310 0x7f42741006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 -- 172.21.15.92:0/1020993742 >> 172.21.15.92:0/1020993742 conn(0x7f4274077210 msgr2=0x7f42740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 -- 172.21.15.92:0/1020993742 shutdown_connections 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 -- 172.21.15.92:0/1020993742 wait complete. 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 Processor -- start 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.554+0000 7f427a699640 1 -- start start 2024-09-11T22:11:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f427a699640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100310 0x7f42741984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f427a699640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274100cc0 0x7f4274198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f427a699640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4274106f30 0x7f427419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f427a699640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4274111ee0 con 0x7f4274100310 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f427a699640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4274111d60 con 0x7f4274106f30 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f427a699640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4274112060 con 0x7f4274100cc0 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100310 0x7f42741984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100310 0x7f42741984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40074/0 (socket says 172.21.15.92:40074) 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 -- 172.21.15.92:0/2252587503 learned_addr learned my addr 172.21.15.92:0/2252587503 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f42737fe640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274100cc0 0x7f4274198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4278c36640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4274106f30 0x7f427419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 -- 172.21.15.92:0/2252587503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274100cc0 msgr2=0x7f4274198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274100cc0 0x7f4274198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 -- 172.21.15.92:0/2252587503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4274106f30 msgr2=0x7f427419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4274106f30 0x7f427419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f4273fff640 1 -- 172.21.15.92:0/2252587503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427419d490 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.555+0000 7f42737fe640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274100cc0 0x7f4274198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.556+0000 7f4273fff640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100310 0x7f42741984e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f426400b4a0 tx=0x7f426400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.556+0000 7f42717fa640 1 -- 172.21.15.92:0/2252587503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264014070 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.556+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427419d780 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.556+0000 7f42717fa640 1 -- 172.21.15.92:0/2252587503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4264010ac0 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.556+0000 7f42717fa640 1 -- 172.21.15.92:0/2252587503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264017c30 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.556+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42741a5060 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.557+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4274100770 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.558+0000 7f42717fa640 1 -- 172.21.15.92:0/2252587503 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4264017070 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.559+0000 7f42717fa640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f425c0a2b80 0x7f425c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.559+0000 7f42737fe640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f425c0a2b80 0x7f425c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.559+0000 7f42717fa640 1 -- 172.21.15.92:0/2252587503 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f42640ce330 con 0x7f4274100310 2024-09-11T22:11:17.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.559+0000 7f42737fe640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f425c0a2b80 0x7f425c0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4268031440 tx=0x7f42680095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:17.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.563+0000 7f42717fa640 1 -- 172.21.15.92:0/2252587503 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f426409b730 con 0x7f4274100310 2024-09-11T22:11:17.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.730+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f427410a140 con 0x7f4274100310 2024-09-11T22:11:17.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.730+0000 7f42717fa640 1 -- 172.21.15.92:0/2252587503 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f42640bb0f0 con 0x7f4274100310 2024-09-11T22:11:17.732 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.732+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f425c0a2b80 msgr2=0x7f425c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.732+0000 7f427a699640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f425c0a2b80 0x7f425c0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4268031440 tx=0x7f42680095f0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.732+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100310 msgr2=0x7f42741984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.732+0000 7f427a699640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100310 0x7f42741984e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f426400b4a0 tx=0x7f426400b960 comp rx=0 tx=0).stop 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 shutdown_connections 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f425c0a2b80 0x7f425c0a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4274106f30 0x7f427419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274100cc0 0x7f4274198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 --2- 172.21.15.92:0/2252587503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4274100310 0x7f42741984e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 >> 172.21.15.92:0/2252587503 conn(0x7f4274077210 msgr2=0x7f42741052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 shutdown_connections 2024-09-11T22:11:17.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:17.733+0000 7f427a699640 1 -- 172.21.15.92:0/2252587503 wait complete. 2024-09-11T22:11:18.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:17 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2252587503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:11:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:17 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2252587503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:11:18.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:17 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2252587503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.a on smithi092 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.d on smithi092 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.g on smithi092 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.j on smithi092 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.b on smithi110 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.e on smithi110 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.h on smithi110 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.k on smithi110 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.c on smithi153 2024-09-11T22:11:18.397 INFO:tasks.cephadm:Adding mds.f on smithi153 2024-09-11T22:11:18.398 INFO:tasks.cephadm:Adding mds.i on smithi153 2024-09-11T22:11:18.398 INFO:tasks.cephadm:Adding mds.l on smithi153 2024-09-11T22:11:18.398 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l' 2024-09-11T22:11:18.656 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:11:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:18 smithi153 ceph-mon[35045]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:19.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.147+0000 7f0e9f267640 1 -- 172.21.15.153:0/3138281532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e981026d0 msgr2=0x7f0e98108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:19.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.147+0000 7f0e9f267640 1 --2- 172.21.15.153:0/3138281532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e981026d0 0x7f0e98108f60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009ca0 tx=0x7f0e9002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 -- 172.21.15.153:0/3138281532 shutdown_connections 2024-09-11T22:11:19.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 --2- 172.21.15.153:0/3138281532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e981026d0 0x7f0e98108f60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 --2- 172.21.15.153:0/3138281532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e98101d10 0x7f0e98102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 --2- 172.21.15.153:0/3138281532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e98100b10 0x7f0e98100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 -- 172.21.15.153:0/3138281532 >> 172.21.15.153:0/3138281532 conn(0x7f0e980fc2a0 msgr2=0x7f0e980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 -- 172.21.15.153:0/3138281532 shutdown_connections 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 -- 172.21.15.153:0/3138281532 wait complete. 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 Processor -- start 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 -- start start 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e98100b10 0x7f0e98197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e98101d10 0x7f0e98198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.148+0000 7f0e9f267640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e981026d0 0x7f0e9819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9f267640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9810c620 con 0x7f0e981026d0 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9f267640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9810c4a0 con 0x7f0e98101d10 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9f267640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9810c7a0 con 0x7f0e98100b10 2024-09-11T22:11:19.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e98100b10 0x7f0e98197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e98100b10 0x7f0e98197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48444/0 (socket says 172.21.15.153:48444) 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 -- 172.21.15.153:0/125238280 learned_addr learned my addr 172.21.15.153:0/125238280 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d804640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e981026d0 0x7f0e9819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9c802640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e98101d10 0x7f0e98198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 -- 172.21.15.153:0/125238280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e98101d10 msgr2=0x7f0e98198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e98101d10 0x7f0e98198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 -- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e981026d0 msgr2=0x7f0e9819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e981026d0 0x7f0e9819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 -- 172.21.15.153:0/125238280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9819fa20 con 0x7f0e98100b10 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d804640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e981026d0 0x7f0e9819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9d003640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e98100b10 0x7f0e98197df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800e9c0 tx=0x7f0e8800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e8e7fc640 1 -- 172.21.15.153:0/125238280 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800cd80 con 0x7f0e98100b10 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e8e7fc640 1 -- 172.21.15.153:0/125238280 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e880035f0 con 0x7f0e98100b10 2024-09-11T22:11:19.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e8e7fc640 1 -- 172.21.15.153:0/125238280 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88010690 con 0x7f0e98100b10 2024-09-11T22:11:19.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9819fd10 con 0x7f0e98100b10 2024-09-11T22:11:19.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.149+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e981a01e0 con 0x7f0e98100b10 2024-09-11T22:11:19.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.151+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e981a0470 con 0x7f0e98100b10 2024-09-11T22:11:19.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.152+0000 7f0e8e7fc640 1 -- 172.21.15.153:0/125238280 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0e880037a0 con 0x7f0e98100b10 2024-09-11T22:11:19.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.153+0000 7f0e8e7fc640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0e840a2c30 0x7f0e840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:19.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.153+0000 7f0e8e7fc640 1 -- 172.21.15.153:0/125238280 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f0e880cefa0 con 0x7f0e98100b10 2024-09-11T22:11:19.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.153+0000 7f0e9c802640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0e840a2c30 0x7f0e840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:19.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.153+0000 7f0e9c802640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0e840a2c30 0x7f0e840a50f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e94005fd0 tx=0x7f0e94005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:19.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.157+0000 7f0e8e7fc640 1 -- 172.21.15.153:0/125238280 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e8809c3a0 con 0x7f0e98100b10 2024-09-11T22:11:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:18 smithi110 ceph-mon[34830]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:18 smithi092 ceph-mon[31173]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:19.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.336+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}) -- 0x7f0e9806b010 con 0x7f0e840a2c30 2024-09-11T22:11:19.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.344+0000 7f0e8e7fc640 1 -- 172.21.15.153:0/125238280 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0e9806b010 con 0x7f0e840a2c30 2024-09-11T22:11:19.345 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mds.all update... 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0e840a2c30 msgr2=0x7f0e840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0e840a2c30 0x7f0e840a50f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e94005fd0 tx=0x7f0e94005850 comp rx=0 tx=0).stop 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e98100b10 msgr2=0x7f0e98197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e98100b10 0x7f0e98197df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800e9c0 tx=0x7f0e8800ee80 comp rx=0 tx=0).stop 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 shutdown_connections 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0e840a2c30 0x7f0e840a50f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e981026d0 0x7f0e9819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e98101d10 0x7f0e98198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 --2- 172.21.15.153:0/125238280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e98100b10 0x7f0e98197df0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 >> 172.21.15.153:0/125238280 conn(0x7f0e980fc2a0 msgr2=0x7f0e9806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 shutdown_connections 2024-09-11T22:11:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-11T22:11:19.346+0000 7f0e9f267640 1 -- 172.21.15.153:0/125238280 wait complete. 2024-09-11T22:11:20.043 DEBUG:teuthology.orchestra.run.smithi092:mds.a> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T22:11:20.046 DEBUG:teuthology.orchestra.run.smithi092:mds.d> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T22:11:20.089 DEBUG:teuthology.orchestra.run.smithi092:mds.g> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T22:11:20.132 DEBUG:teuthology.orchestra.run.smithi092:mds.j> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T22:11:20.174 DEBUG:teuthology.orchestra.run.smithi110:mds.b> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T22:11:20.177 DEBUG:teuthology.orchestra.run.smithi110:mds.e> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T22:11:20.221 DEBUG:teuthology.orchestra.run.smithi110:mds.h> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T22:11:20.264 DEBUG:teuthology.orchestra.run.smithi110:mds.k> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T22:11:20.307 DEBUG:teuthology.orchestra.run.smithi153:mds.c> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T22:11:20.309 DEBUG:teuthology.orchestra.run.smithi153:mds.f> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T22:11:20.353 DEBUG:teuthology.orchestra.run.smithi153:mds.i> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T22:11:20.396 DEBUG:teuthology.orchestra.run.smithi153:mds.l> sudo journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T22:11:20.439 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T22:11:20.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:20.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:20.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:20 smithi110 ceph-mon[34830]: Deploying daemon mds.l on smithi153 2024-09-11T22:11:20.494 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:11:20.494 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-11T22:11:20.494 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:20.494 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:20.494 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:20.495 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:20.495 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:20.495 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:20.495 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:20.495 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:20.495 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:20.495 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:20 smithi153 ceph-mon[35045]: Deploying daemon mds.l on smithi153 2024-09-11T22:11:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:20.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:11:20.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:20.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:20 smithi092 ceph-mon[31173]: Deploying daemon mds.l on smithi153 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.152+0000 7f177574a640 1 -- 172.21.15.92:0/1278331130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074510 msgr2=0x7f17700748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.152+0000 7f177574a640 1 --2- 172.21.15.92:0/1278331130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074510 0x7f17700748f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1760009900 tx=0x7f176002a480 comp rx=0 tx=0).stop 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.152+0000 7f177574a640 1 -- 172.21.15.92:0/1278331130 shutdown_connections 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.152+0000 7f177574a640 1 --2- 172.21.15.92:0/1278331130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f177010e440 0x7f1770110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.152+0000 7f177574a640 1 --2- 172.21.15.92:0/1278331130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1770074e30 0x7f177010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.152+0000 7f177574a640 1 --2- 172.21.15.92:0/1278331130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074510 0x7f17700748f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.152+0000 7f177574a640 1 -- 172.21.15.92:0/1278331130 >> 172.21.15.92:0/1278331130 conn(0x7f17700fd8c0 msgr2=0x7f17700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 -- 172.21.15.92:0/1278331130 shutdown_connections 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 -- 172.21.15.92:0/1278331130 wait complete. 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 Processor -- start 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 -- start start 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1770074510 0x7f177019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074e30 0x7f177019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f177010e440 0x7f177019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17701140b0 con 0x7f1770074e30 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1770113f30 con 0x7f177010e440 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.153+0000 7f177574a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1770114230 con 0x7f1770074510 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074e30 0x7f177019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074e30 0x7f177019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40094/0 (socket says 172.21.15.92:40094) 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 -- 172.21.15.92:0/1144011191 learned_addr learned my addr 172.21.15.92:0/1144011191 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f176effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1770074510 0x7f177019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 -- 172.21.15.92:0/1144011191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1770074510 msgr2=0x7f177019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1770074510 0x7f177019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 -- 172.21.15.92:0/1144011191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f177010e440 msgr2=0x7f177019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f176f7fe640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f177010e440 0x7f177019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f177010e440 0x7f177019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 -- 172.21.15.92:0/1144011191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177019f660 con 0x7f1770074e30 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f176f7fe640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f177010e440 0x7f177019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f17667fc640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074e30 0x7f177019abf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f175800d770 tx=0x7f175800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f176cff9640 1 -- 172.21.15.92:0/1144011191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758017070 con 0x7f1770074e30 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177019f950 con 0x7f1770074e30 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f176cff9640 1 -- 172.21.15.92:0/1144011191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17580054d0 con 0x7f1770074e30 2024-09-11T22:11:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f176cff9640 1 -- 172.21.15.92:0/1144011191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758004cf0 con 0x7f1770074e30 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.154+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17701a7230 con 0x7f1770074e30 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.155+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17700748f0 con 0x7f1770074e30 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.156+0000 7f176cff9640 1 -- 172.21.15.92:0/1144011191 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1758004e90 con 0x7f1770074e30 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.157+0000 7f176cff9640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f175c0a2b80 0x7f175c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.157+0000 7f176effd640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f175c0a2b80 0x7f175c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.157+0000 7f176cff9640 1 -- 172.21.15.92:0/1144011191 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f17580ce610 con 0x7f1770074e30 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.157+0000 7f176effd640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f175c0a2b80 0x7f175c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1760031440 tx=0x7f1760005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.161+0000 7f176cff9640 1 -- 172.21.15.92:0/1144011191 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f175809ba10 con 0x7f1770074e30 2024-09-11T22:11:21.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.362+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17701095c0 con 0x7f1770074e30 2024-09-11T22:11:21.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.362+0000 7f176cff9640 1 -- 172.21.15.92:0/1144011191 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f175809b310 con 0x7f1770074e30 2024-09-11T22:11:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:21.364 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":1,"btime":"2024-09-11T22:04:29:869139+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T22:11:21.364 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 1 2024-09-11T22:11:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f175c0a2b80 msgr2=0x7f175c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f175c0a2b80 0x7f175c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1760031440 tx=0x7f1760005b50 comp rx=0 tx=0).stop 2024-09-11T22:11:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074e30 msgr2=0x7f177019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074e30 0x7f177019abf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f175800d770 tx=0x7f175800dc30 comp rx=0 tx=0).stop 2024-09-11T22:11:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 shutdown_connections 2024-09-11T22:11:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f175c0a2b80 0x7f175c0a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f177010e440 0x7f177019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1770074e30 0x7f177019abf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 --2- 172.21.15.92:0/1144011191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1770074510 0x7f177019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 >> 172.21.15.92:0/1144011191 conn(0x7f17700fd8c0 msgr2=0x7f17700ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 shutdown_connections 2024-09-11T22:11:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:21.365+0000 7f177574a640 1 -- 172.21.15.92:0/1144011191 wait complete. 2024-09-11T22:11:21.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:21 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1144011191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:21.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:21 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1144011191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:21.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:21 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1144011191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:21.962 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:21 smithi153 systemd[1]: Starting Ceph mds.l for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:22.254 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:21 smithi153 podman[64747]: 2024-09-11 22:11:21.864510813 +0000 UTC m=+0.015618298 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:22.254 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:22 smithi153 podman[64747]: 2024-09-11 22:11:22.001352768 +0000 UTC m=+0.152460248 container create d1a8ee70b076e97c301cea4c18d800b416704d6701bb3bdd9496a31345f1be65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T22:11:22.254 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:22 smithi153 podman[64747]: 2024-09-11 22:11:22.254142978 +0000 UTC m=+0.405250456 container init d1a8ee70b076e97c301cea4c18d800b416704d6701bb3bdd9496a31345f1be65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T22:11:22.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:22 smithi153 ceph-mon[35045]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:22 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6832/490387556,v1:172.21.15.153:6833/490387556] up:boot 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:22 smithi153 ceph-mon[35045]: fsmap 1 up:standby 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:22 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:22 smithi153 podman[64747]: 2024-09-11 22:11:22.25720754 +0000 UTC m=+0.408315027 container start d1a8ee70b076e97c301cea4c18d800b416704d6701bb3bdd9496a31345f1be65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:22 smithi153 ceph-mds[64766]: -- 172.21.15.153:0/294747332 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a77039860 con 0x560a77017000 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:22 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l[64762]: starting mds.l at 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:22 smithi153 bash[64747]: d1a8ee70b076e97c301cea4c18d800b416704d6701bb3bdd9496a31345f1be65 2024-09-11T22:11:22.539 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:11:22 smithi153 systemd[1]: Started Ceph mds.l for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:22 smithi110 ceph-mon[34830]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:22 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6832/490387556,v1:172.21.15.153:6833/490387556] up:boot 2024-09-11T22:11:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:22 smithi110 ceph-mon[34830]: fsmap 1 up:standby 2024-09-11T22:11:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:11:22.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:22 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:22.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:22 smithi092 ceph-mon[31173]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:22.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:22 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6832/490387556,v1:172.21.15.153:6833/490387556] up:boot 2024-09-11T22:11:22.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:22 smithi092 ceph-mon[31173]: fsmap 1 up:standby 2024-09-11T22:11:22.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:11:22.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:22 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:23.422 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:23.422 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:23.422 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:23.422 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:23.422 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:23 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:23.422 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:23 smithi110 ceph-mon[34830]: Deploying daemon mds.k on smithi110 2024-09-11T22:11:23.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:23.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:23.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:23.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:23.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:23 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:23.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:23 smithi092 ceph-mon[31173]: Deploying daemon mds.k on smithi110 2024-09-11T22:11:23.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:23.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:23.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:23.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:23.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:23 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:23.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:23 smithi153 ceph-mon[35045]: Deploying daemon mds.k on smithi110 2024-09-11T22:11:24.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:24.622 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:24 smithi110 ceph-mon[34830]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:24.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:24 smithi092 ceph-mon[31173]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:24.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:24 smithi153 ceph-mon[35045]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 -- 172.21.15.92:0/2915547702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264105590 msgr2=0x7fc26410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 --2- 172.21.15.92:0/2915547702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264105590 0x7fc26410d110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009ca0 tx=0x7fc25c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:11:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 -- 172.21.15.92:0/2915547702 shutdown_connections 2024-09-11T22:11:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 --2- 172.21.15.92:0/2915547702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264105590 0x7fc26410d110 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 --2- 172.21.15.92:0/2915547702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc264100e10 0x7fc264104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 --2- 172.21.15.92:0/2915547702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc264100460 0x7fc264100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 -- 172.21.15.92:0/2915547702 >> 172.21.15.92:0/2915547702 conn(0x7fc2640fbee0 msgr2=0x7fc2640fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 -- 172.21.15.92:0/2915547702 shutdown_connections 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.853+0000 7fc2693b1640 1 -- 172.21.15.92:0/2915547702 wait complete. 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 Processor -- start 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 -- start start 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264100460 0x7fc264198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc264100e10 0x7fc264198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc264105590 0x7fc26419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc264110d20 con 0x7fc264100460 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc264110ba0 con 0x7fc264105590 2024-09-11T22:11:24.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2693b1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc264110ea0 con 0x7fc264100e10 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc262ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264100460 0x7fc264198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc262ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264100460 0x7fc264198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40112/0 (socket says 172.21.15.92:40112) 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc262ffd640 1 -- 172.21.15.92:0/3077430362 learned_addr learned my addr 172.21.15.92:0/3077430362 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.854+0000 7fc2637fe640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc264105590 0x7fc26419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc2627fc640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc264100e10 0x7fc264198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc262ffd640 1 -- 172.21.15.92:0/3077430362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc264100e10 msgr2=0x7fc264198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc262ffd640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc264100e10 0x7fc264198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc262ffd640 1 -- 172.21.15.92:0/3077430362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc264105590 msgr2=0x7fc26419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc262ffd640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc264105590 0x7fc26419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc262ffd640 1 -- 172.21.15.92:0/3077430362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26419d300 con 0x7fc264100460 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc2627fc640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc264100e10 0x7fc264198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc262ffd640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264100460 0x7fc264198350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc25400d770 tx=0x7fc25400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc24bfff640 1 -- 172.21.15.92:0/3077430362 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254017070 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc24bfff640 1 -- 172.21.15.92:0/3077430362 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2540054d0 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26419d5f0 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc24bfff640 1 -- 172.21.15.92:0/3077430362 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254004cf0 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.855+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2641a4ed0 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.856+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc264100840 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.857+0000 7fc24bfff640 1 -- 172.21.15.92:0/3077430362 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc254004e90 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.858+0000 7fc24bfff640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc24c0a2b80 0x7fc24c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.858+0000 7fc2627fc640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc24c0a2b80 0x7fc24c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.858+0000 7fc24bfff640 1 -- 172.21.15.92:0/3077430362 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc2540ce610 con 0x7fc264100460 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.858+0000 7fc2627fc640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc24c0a2b80 0x7fc24c0a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc258005fd0 tx=0x7fc2580094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:24.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:24.863+0000 7fc24bfff640 1 -- 172.21.15.92:0/3077430362 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc25409ba10 con 0x7fc264100460 2024-09-11T22:11:25.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.064+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc26410a780 con 0x7fc264100460 2024-09-11T22:11:25.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.064+0000 7fc24bfff640 1 -- 172.21.15.92:0/3077430362 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fc25409b310 con 0x7fc264100460 2024-09-11T22:11:25.066 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:25.066 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"btime":"2024-09-11T22:11:22:285646+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T22:11:25.066 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 2 2024-09-11T22:11:25.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc24c0a2b80 msgr2=0x7fc24c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:25.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc24c0a2b80 0x7fc24c0a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc258005fd0 tx=0x7fc2580094d0 comp rx=0 tx=0).stop 2024-09-11T22:11:25.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264100460 msgr2=0x7fc264198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:25.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264100460 0x7fc264198350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc25400d770 tx=0x7fc25400dc30 comp rx=0 tx=0).stop 2024-09-11T22:11:25.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 shutdown_connections 2024-09-11T22:11:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc24c0a2b80 0x7fc24c0a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc264105590 0x7fc26419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc264100e10 0x7fc264198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 --2- 172.21.15.92:0/3077430362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc264100460 0x7fc264198350 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 >> 172.21.15.92:0/3077430362 conn(0x7fc2640fbee0 msgr2=0x7fc2640fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 shutdown_connections 2024-09-11T22:11:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:25.068+0000 7fc2693b1640 1 -- 172.21.15.92:0/3077430362 wait complete. 2024-09-11T22:11:25.217 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:11:25 smithi110 systemd[1]: Started Ceph mds.k for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:25.690 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:25 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3077430362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:25.690 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:25 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:25 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:25 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:25 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:25.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:25 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:25.691 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:25 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:25 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3077430362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:25 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:25 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:25 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:25 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:25 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:25.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:25 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:25.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:25 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3077430362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:25.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:25 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:25 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:25 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:25.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:25 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:25.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:25 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:25.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:25 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:26.703 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:26 smithi153 ceph-mon[35045]: Deploying daemon mds.j on smithi092 2024-09-11T22:11:26.703 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:26 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6832/797341789,v1:172.21.15.110:6833/797341789] up:boot 2024-09-11T22:11:26.703 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:26 smithi153 ceph-mon[35045]: fsmap 2 up:standby 2024-09-11T22:11:26.703 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:26 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:11:26.703 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:26 smithi153 ceph-mon[35045]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:26 smithi110 ceph-mon[34830]: Deploying daemon mds.j on smithi092 2024-09-11T22:11:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:26 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6832/797341789,v1:172.21.15.110:6833/797341789] up:boot 2024-09-11T22:11:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:26 smithi110 ceph-mon[34830]: fsmap 2 up:standby 2024-09-11T22:11:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:26 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:11:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:26 smithi110 ceph-mon[34830]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:26.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:26 smithi092 ceph-mon[31173]: Deploying daemon mds.j on smithi092 2024-09-11T22:11:26.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:26 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6832/797341789,v1:172.21.15.110:6833/797341789] up:boot 2024-09-11T22:11:26.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:26 smithi092 ceph-mon[31173]: fsmap 2 up:standby 2024-09-11T22:11:26.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:26 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:11:26.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:26 smithi092 ceph-mon[31173]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:27.664 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:27 smithi092 systemd[1]: Starting Ceph mds.j for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:27.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:28.018 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:27 smithi092 podman[71333]: 2024-09-11 22:11:27.566986859 +0000 UTC m=+0.015873311 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:28.018 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:27 smithi092 podman[71333]: 2024-09-11 22:11:27.693384198 +0000 UTC m=+0.142270645 container create 6535b833ed09aa08dd979cfd2477cd8b1b0dfe2424aefb335dee36e41c4e3893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T22:11:28.265 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:28.387 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:28 smithi092 podman[71333]: 2024-09-11 22:11:28.097026821 +0000 UTC m=+0.545913259 container init 6535b833ed09aa08dd979cfd2477cd8b1b0dfe2424aefb335dee36e41c4e3893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T22:11:28.387 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:28 smithi092 podman[71333]: 2024-09-11 22:11:28.100346876 +0000 UTC m=+0.549233314 container start 6535b833ed09aa08dd979cfd2477cd8b1b0dfe2424aefb335dee36e41c4e3893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T22:11:28.387 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mds[71382]: -- 172.21.15.92:0/3321291077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5619c259f860 con 0x5619c257d400 2024-09-11T22:11:28.387 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j[71348]: starting mds.j at 2024-09-11T22:11:28.387 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:28 smithi092 bash[71333]: 6535b833ed09aa08dd979cfd2477cd8b1b0dfe2424aefb335dee36e41c4e3893 2024-09-11T22:11:28.387 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:11:28 smithi092 systemd[1]: Started Ceph mds.j for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:28.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 -- 172.21.15.92:0/635271734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 msgr2=0x7f587010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:28.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 --2- 172.21.15.92:0/635271734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587010da40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5868009ca0 tx=0x7f586802a950 comp rx=0 tx=0).stop 2024-09-11T22:11:28.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 -- 172.21.15.92:0/635271734 shutdown_connections 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 --2- 172.21.15.92:0/635271734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587010da40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 --2- 172.21.15.92:0/635271734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5870102570 0x7f58701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 --2- 172.21.15.92:0/635271734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5870100e20 0x7f5870101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 -- 172.21.15.92:0/635271734 >> 172.21.15.92:0/635271734 conn(0x7f58700fc7d0 msgr2=0x7f58700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 -- 172.21.15.92:0/635271734 shutdown_connections 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.741+0000 7f587730e640 1 -- 172.21.15.92:0/635271734 wait complete. 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 Processor -- start 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 -- start start 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5870100e20 0x7f5870198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5870102570 0x7f5870198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:28.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5870110ed0 con 0x7f5870102ef0 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5870110d50 con 0x7f5870100e20 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f587730e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5870111050 con 0x7f5870102570 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f58758ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f58758ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59370/0 (socket says 172.21.15.92:59370) 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.742+0000 7f58758ab640 1 -- 172.21.15.92:0/580845136 learned_addr learned my addr 172.21.15.92:0/580845136 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58750aa640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5870100e20 0x7f5870198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58748a9640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5870102570 0x7f5870198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58758ab640 1 -- 172.21.15.92:0/580845136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5870102570 msgr2=0x7f5870198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58758ab640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5870102570 0x7f5870198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58758ab640 1 -- 172.21.15.92:0/580845136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5870100e20 msgr2=0x7f5870198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58758ab640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5870100e20 0x7f5870198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58758ab640 1 -- 172.21.15.92:0/580845136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587019d4f0 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58748a9640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5870102570 0x7f5870198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58750aa640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5870100e20 0x7f5870198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58758ab640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587019ce10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5868006010 tx=0x7f586802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58667fc640 1 -- 172.21.15.92:0/580845136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868034040 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587019d780 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58667fc640 1 -- 172.21.15.92:0/580845136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5868003e00 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f58667fc640 1 -- 172.21.15.92:0/580845136 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868045480 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.743+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701a50c0 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.744+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5870101280 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.745+0000 7f58667fc640 1 -- 172.21.15.92:0/580845136 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5868037020 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.746+0000 7f58667fc640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f585c0a2b80 0x7f585c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.746+0000 7f58750aa640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f585c0a2b80 0x7f585c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.747+0000 7f58667fc640 1 -- 172.21.15.92:0/580845136 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f58680efec0 con 0x7f5870102ef0 2024-09-11T22:11:28.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.747+0000 7f58750aa640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f585c0a2b80 0x7f585c0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f586c009930 tx=0x7f586c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:28.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.750+0000 7f58667fc640 1 -- 172.21.15.92:0/580845136 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58680bd220 con 0x7f5870102ef0 2024-09-11T22:11:28.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mon[31173]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:28.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:28.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:28.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:28.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:28.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:28.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:28 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:28.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.963+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58701998a0 con 0x7f5870102ef0 2024-09-11T22:11:28.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.963+0000 7f58667fc640 1 -- 172.21.15.92:0/580845136 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f58701998a0 con 0x7f5870102ef0 2024-09-11T22:11:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:28.964 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":4,"btime":"2024-09-11T22:11:28:726203+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T22:11:28.964 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 4 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f585c0a2b80 msgr2=0x7f585c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f585c0a2b80 0x7f585c0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f586c009930 tx=0x7f586c008040 comp rx=0 tx=0).stop 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 msgr2=0x7f587019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587019ce10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5868006010 tx=0x7f586802f710 comp rx=0 tx=0).stop 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 shutdown_connections 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f585c0a2b80 0x7f585c0a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5870102ef0 0x7f587019ce10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5870102570 0x7f5870198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 --2- 172.21.15.92:0/580845136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5870100e20 0x7f5870198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 >> 172.21.15.92:0/580845136 conn(0x7f58700fc7d0 msgr2=0x7f587006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 shutdown_connections 2024-09-11T22:11:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:28.966+0000 7f587730e640 1 -- 172.21.15.92:0/580845136 wait complete. 2024-09-11T22:11:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:28 smithi153 ceph-mon[35045]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:29.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:28 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:28 smithi110 ceph-mon[34830]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:28 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:29.996 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:29 smithi092 ceph-mon[31173]: Deploying daemon mds.d on smithi092 2024-09-11T22:11:29.996 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:29 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.92:6834/1683671615,v1:172.21.15.92:6835/1683671615] up:boot 2024-09-11T22:11:29.996 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:29 smithi092 ceph-mon[31173]: fsmap 3 up:standby 2024-09-11T22:11:29.996 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:29 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:11:29.996 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:29 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/580845136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:30.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:29 smithi153 ceph-mon[35045]: Deploying daemon mds.d on smithi092 2024-09-11T22:11:30.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:29 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.92:6834/1683671615,v1:172.21.15.92:6835/1683671615] up:boot 2024-09-11T22:11:30.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:29 smithi153 ceph-mon[35045]: fsmap 3 up:standby 2024-09-11T22:11:30.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:29 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:11:30.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:29 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/580845136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:29 smithi110 ceph-mon[34830]: Deploying daemon mds.d on smithi092 2024-09-11T22:11:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:29 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.92:6834/1683671615,v1:172.21.15.92:6835/1683671615] up:boot 2024-09-11T22:11:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:29 smithi110 ceph-mon[34830]: fsmap 3 up:standby 2024-09-11T22:11:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:29 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:11:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:29 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/580845136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:30.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:30 smithi092 ceph-mon[31173]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:31.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:30 smithi153 ceph-mon[35045]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:30 smithi110 ceph-mon[34830]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:31.272 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 systemd[1]: Starting Ceph mds.d for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:31.597 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 podman[71749]: 2024-09-11 22:11:31.222753649 +0000 UTC m=+0.015422920 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:31.597 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 podman[71749]: 2024-09-11 22:11:31.344620575 +0000 UTC m=+0.137289897 container create 6576448ede5a1f3b70b9642ddbe6b7c73079ffabb37959179b47a270d6434128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:11:31.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:32.018 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 podman[71749]: 2024-09-11 22:11:31.597199801 +0000 UTC m=+0.389869067 container init 6576448ede5a1f3b70b9642ddbe6b7c73079ffabb37959179b47a270d6434128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T22:11:32.019 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 podman[71749]: 2024-09-11 22:11:31.599933472 +0000 UTC m=+0.392602747 container start 6576448ede5a1f3b70b9642ddbe6b7c73079ffabb37959179b47a270d6434128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:11:32.019 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 ceph-mds[71769]: -- 172.21.15.92:0/1015871473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55caa4863860 con 0x55caa4841000 2024-09-11T22:11:32.019 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d[71765]: starting mds.d at 2024-09-11T22:11:32.019 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 bash[71749]: 6576448ede5a1f3b70b9642ddbe6b7c73079ffabb37959179b47a270d6434128 2024-09-11T22:11:32.019 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:11:31 smithi092 systemd[1]: Started Ceph mds.d for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:32.020 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/806930361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 msgr2=0x7f8e4410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/806930361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4410da40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009ca0 tx=0x7f8e3c02a950 comp rx=0 tx=0).stop 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/806930361 shutdown_connections 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/806930361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4410da40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/806930361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e44102570 0x7f8e441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/806930361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e44100e20 0x7f8e44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/806930361 >> 172.21.15.92:0/806930361 conn(0x7f8e440fc7d0 msgr2=0x7f8e440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/806930361 shutdown_connections 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/806930361 wait complete. 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.515+0000 7f8e4a0b3640 1 Processor -- start 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e4a0b3640 1 -- start start 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e4a0b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e44100e20 0x7f8e44197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:32.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e4a0b3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e44102570 0x7f8e44197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e4a0b3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e4a0b3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44110e30 con 0x7f8e44102ef0 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e4a0b3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44110cb0 con 0x7f8e44100e20 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e4a0b3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44110fb0 con 0x7f8e44102570 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59402/0 (socket says 172.21.15.92:59402) 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 -- 172.21.15.92:0/1362219453 learned_addr learned my addr 172.21.15.92:0/1362219453 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e437fe640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e44100e20 0x7f8e44197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e3affd640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e44102570 0x7f8e44197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 -- 172.21.15.92:0/1362219453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e44102570 msgr2=0x7f8e44197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e44102570 0x7f8e44197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 -- 172.21.15.92:0/1362219453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e44100e20 msgr2=0x7f8e44197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e44100e20 0x7f8e44197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.516+0000 7f8e43fff640 1 -- 172.21.15.92:0/1362219453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4419c1c0 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.517+0000 7f8e43fff640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4419bae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c02ae70 tx=0x7f8e3c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.517+0000 7f8e417fa640 1 -- 172.21.15.92:0/1362219453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c034040 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.517+0000 7f8e417fa640 1 -- 172.21.15.92:0/1362219453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e3c0308f0 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.517+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4419c450 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.517+0000 7f8e417fa640 1 -- 172.21.15.92:0/1362219453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c004780 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.517+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e441a3d90 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.518+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e44101200 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.519+0000 7f8e417fa640 1 -- 172.21.15.92:0/1362219453 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8e3c037020 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.520+0000 7f8e417fa640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e300a2b80 0x7f8e300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.520+0000 7f8e437fe640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e300a2b80 0x7f8e300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.520+0000 7f8e417fa640 1 -- 172.21.15.92:0/1362219453 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8e3c0efcf0 con 0x7f8e44102ef0 2024-09-11T22:11:32.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.520+0000 7f8e437fe640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e300a2b80 0x7f8e300a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400bbe0 tx=0x7f8e3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.524+0000 7f8e417fa640 1 -- 172.21.15.92:0/1362219453 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e3c0f4070 con 0x7f8e44102ef0 2024-09-11T22:11:32.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.721+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e441a3f60 con 0x7f8e44102ef0 2024-09-11T22:11:32.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.721+0000 7f8e417fa640 1 -- 172.21.15.92:0/1362219453 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f8e3c0bd0f0 con 0x7f8e44102ef0 2024-09-11T22:11:32.722 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:32.722 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"btime":"2024-09-11T22:11:31:738871+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T22:11:32.722 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 5 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e300a2b80 msgr2=0x7f8e300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e300a2b80 0x7f8e300a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400bbe0 tx=0x7f8e3400b390 comp rx=0 tx=0).stop 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 msgr2=0x7f8e4419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4419bae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c02ae70 tx=0x7f8e3c02f5d0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 shutdown_connections 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e300a2b80 0x7f8e300a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e44102ef0 0x7f8e4419bae0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e44102570 0x7f8e44197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 --2- 172.21.15.92:0/1362219453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e44100e20 0x7f8e44197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 >> 172.21.15.92:0/1362219453 conn(0x7f8e440fc7d0 msgr2=0x7f8e44106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 shutdown_connections 2024-09-11T22:11:32.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:32.724+0000 7f8e4a0b3640 1 -- 172.21.15.92:0/1362219453 wait complete. 2024-09-11T22:11:33.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.92:6836/2477546203,v1:172.21.15.92:6837/2477546203] up:boot 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: fsmap 4 up:standby 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: Deploying daemon mds.e on smithi110 2024-09-11T22:11:33.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:32 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1362219453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:33.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:33.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.92:6836/2477546203,v1:172.21.15.92:6837/2477546203] up:boot 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: fsmap 4 up:standby 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: Deploying daemon mds.e on smithi110 2024-09-11T22:11:33.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:32 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1362219453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:33.199 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:33.199 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.92:6836/2477546203,v1:172.21.15.92:6837/2477546203] up:boot 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: fsmap 4 up:standby 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: Deploying daemon mds.e on smithi110 2024-09-11T22:11:33.200 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:32 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1362219453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:33.888 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:33 smithi110 systemd[1]: Starting Ceph mds.e for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:34.171 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:33 smithi110 podman[64524]: 2024-09-11 22:11:33.790739528 +0000 UTC m=+0.015853150 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:34.171 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:33 smithi110 podman[64524]: 2024-09-11 22:11:33.910309377 +0000 UTC m=+0.135422993 container create e28c89905fea9ccf2a95b7d6042cd0daa5b5b16dc4ac72244ec649d4f618064b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T22:11:34.171 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:34 smithi110 podman[64524]: 2024-09-11 22:11:34.171266164 +0000 UTC m=+0.396379783 container init e28c89905fea9ccf2a95b7d6042cd0daa5b5b16dc4ac72244ec649d4f618064b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T22:11:34.467 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:34 smithi110 podman[64524]: 2024-09-11 22:11:34.174196115 +0000 UTC m=+0.399309729 container start e28c89905fea9ccf2a95b7d6042cd0daa5b5b16dc4ac72244ec649d4f618064b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T22:11:34.467 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mds[64543]: -- 172.21.15.110:0/3354190499 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dac3a39860 con 0x55dac3a16800 2024-09-11T22:11:34.467 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e[64539]: starting mds.e at 2024-09-11T22:11:34.467 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:34 smithi110 bash[64524]: e28c89905fea9ccf2a95b7d6042cd0daa5b5b16dc4ac72244ec649d4f618064b 2024-09-11T22:11:34.467 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:11:34 smithi110 systemd[1]: Started Ceph mds.e for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:35.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:34 smithi092 ceph-mon[31173]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:35.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:34 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:35.031 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:34 smithi153 ceph-mon[35045]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:35.031 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.031 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.031 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.031 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:35.032 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:35.032 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:34 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mon[34830]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:34 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:35.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:36.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:35 smithi092 ceph-mon[31173]: Deploying daemon mds.f on smithi153 2024-09-11T22:11:36.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:35 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6834/1455978865,v1:172.21.15.110:6835/1455978865] up:boot 2024-09-11T22:11:36.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:35 smithi092 ceph-mon[31173]: fsmap 5 up:standby 2024-09-11T22:11:36.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:35 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:35 smithi153 ceph-mon[35045]: Deploying daemon mds.f on smithi153 2024-09-11T22:11:36.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:35 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6834/1455978865,v1:172.21.15.110:6835/1455978865] up:boot 2024-09-11T22:11:36.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:35 smithi153 ceph-mon[35045]: fsmap 5 up:standby 2024-09-11T22:11:36.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:35 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:11:36.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:35 smithi110 ceph-mon[34830]: Deploying daemon mds.f on smithi153 2024-09-11T22:11:36.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:35 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6834/1455978865,v1:172.21.15.110:6835/1455978865] up:boot 2024-09-11T22:11:36.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:35 smithi110 ceph-mon[34830]: fsmap 5 up:standby 2024-09-11T22:11:36.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:35 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:11:36.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 -- 172.21.15.92:0/1914746587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514101de0 msgr2=0x7f25141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:36.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 --2- 172.21.15.92:0/1914746587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514101de0 0x7f25141021c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2504009900 tx=0x7f250402a410 comp rx=0 tx=0).stop 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 -- 172.21.15.92:0/1914746587 shutdown_connections 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 --2- 172.21.15.92:0/1914746587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514106f10 0x7f251410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 --2- 172.21.15.92:0/1914746587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2514102790 0x7f25141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 --2- 172.21.15.92:0/1914746587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514101de0 0x7f25141021c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 -- 172.21.15.92:0/1914746587 >> 172.21.15.92:0/1914746587 conn(0x7f25140fd8c0 msgr2=0x7f25140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 -- 172.21.15.92:0/1914746587 shutdown_connections 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 -- 172.21.15.92:0/1914746587 wait complete. 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 Processor -- start 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.223+0000 7f25199ca640 1 -- start start 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514101de0 0x7f25141983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514102790 0x7f2514198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2514106f10 0x7f251419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2514111e20 con 0x7f2514102790 2024-09-11T22:11:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2514111ca0 con 0x7f2514101de0 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2514111fa0 con 0x7f2514106f10 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514102790 0x7f2514198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514102790 0x7f2514198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44376/0 (socket says 172.21.15.92:44376) 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 -- 172.21.15.92:0/458489291 learned_addr learned my addr 172.21.15.92:0/458489291 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25137fe640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2514106f10 0x7f251419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f2512ffd640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514101de0 0x7f25141983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 -- 172.21.15.92:0/458489291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2514106f10 msgr2=0x7f251419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2514106f10 0x7f251419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 -- 172.21.15.92:0/458489291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514101de0 msgr2=0x7f25141983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514101de0 0x7f25141983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 -- 172.21.15.92:0/458489291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251419d390 con 0x7f2514102790 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f250bfff640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514102790 0x7f2514198920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00e9f0 tx=0x7f24fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25189c8640 1 -- 172.21.15.92:0/458489291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00cd80 con 0x7f2514102790 2024-09-11T22:11:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25189c8640 1 -- 172.21.15.92:0/458489291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24fc0054d0 con 0x7f2514102790 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251419d680 con 0x7f2514102790 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25189c8640 1 -- 172.21.15.92:0/458489291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc010490 con 0x7f2514102790 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.224+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141a4f60 con 0x7f2514102790 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.225+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25141021c0 con 0x7f2514102790 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.227+0000 7f25189c8640 1 -- 172.21.15.92:0/458489291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f24fc005000 con 0x7f2514102790 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.227+0000 7f25189c8640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f25000a2b80 0x7f25000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.227+0000 7f2512ffd640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f25000a2b80 0x7f25000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.228+0000 7f25189c8640 1 -- 172.21.15.92:0/458489291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f24fc0cf120 con 0x7f2514102790 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.228+0000 7f2512ffd640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f25000a2b80 0x7f25000a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2504031440 tx=0x7f2504005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.230+0000 7f25189c8640 1 -- 172.21.15.92:0/458489291 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24fc0541b0 con 0x7f2514102790 2024-09-11T22:11:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.434+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f251410a150 con 0x7f2514102790 2024-09-11T22:11:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.435+0000 7f25189c8640 1 -- 172.21.15.92:0/458489291 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4738 (secure 0 0 0) 0x7f24fc0052f0 con 0x7f2514102790 2024-09-11T22:11:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:36.435 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"btime":"2024-09-11T22:11:34:739368+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T22:11:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2024-09-11T22:11:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f25000a2b80 msgr2=0x7f25000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f25000a2b80 0x7f25000a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2504031440 tx=0x7f2504005b50 comp rx=0 tx=0).stop 2024-09-11T22:11:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514102790 msgr2=0x7f2514198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514102790 0x7f2514198920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00e9f0 tx=0x7f24fc00edf0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 shutdown_connections 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f25000a2b80 0x7f25000a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2514106f10 0x7f251419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514102790 0x7f2514198920 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 --2- 172.21.15.92:0/458489291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514101de0 0x7f25141983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 >> 172.21.15.92:0/458489291 conn(0x7f25140fd8c0 msgr2=0x7f25140feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 shutdown_connections 2024-09-11T22:11:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:36.437+0000 7f25199ca640 1 -- 172.21.15.92:0/458489291 wait complete. 2024-09-11T22:11:36.514 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 systemd[1]: Starting Ceph mds.f for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:36.771 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:36 smithi153 ceph-mon[35045]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:36.771 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:36 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/458489291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:36.771 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 podman[65052]: 2024-09-11 22:11:36.416462267 +0000 UTC m=+0.015829088 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:36.771 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 podman[65052]: 2024-09-11 22:11:36.538177681 +0000 UTC m=+0.137544535 container create 9c7c2db94ea0492648e76835b98daf9782cbead66e1142c10fa81f44e1e1248d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:11:37.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:36 smithi092 ceph-mon[31173]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:37.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:36 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/458489291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:37.039 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 podman[65052]: 2024-09-11 22:11:36.78721023 +0000 UTC m=+0.386577049 container init 9c7c2db94ea0492648e76835b98daf9782cbead66e1142c10fa81f44e1e1248d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:11:37.039 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 podman[65052]: 2024-09-11 22:11:36.790165828 +0000 UTC m=+0.389532637 container start 9c7c2db94ea0492648e76835b98daf9782cbead66e1142c10fa81f44e1e1248d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-11T22:11:37.039 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 ceph-mds[65072]: -- 172.21.15.153:0/4114387134 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56303c81d860 con 0x56303c7fa800 2024-09-11T22:11:37.039 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f[65067]: starting mds.f at 2024-09-11T22:11:37.039 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 bash[65052]: 9c7c2db94ea0492648e76835b98daf9782cbead66e1142c10fa81f44e1e1248d 2024-09-11T22:11:37.039 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:11:36 smithi153 systemd[1]: Started Ceph mds.f for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:36 smithi110 ceph-mon[34830]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:36 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/458489291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: Deploying daemon mds.c on smithi153 2024-09-11T22:11:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:38.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] up:boot 2024-09-11T22:11:38.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: fsmap 6 up:standby 2024-09-11T22:11:38.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:37 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:11:38.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: Deploying daemon mds.c on smithi153 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] up:boot 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: fsmap 6 up:standby 2024-09-11T22:11:38.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:37 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: Deploying daemon mds.c on smithi153 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] up:boot 2024-09-11T22:11:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: fsmap 6 up:standby 2024-09-11T22:11:38.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:37 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:11:39.039 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:38 smithi153 systemd[1]: Starting Ceph mds.c for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:39.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:39.357 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 podman[65354]: 2024-09-11 22:11:38.967486992 +0000 UTC m=+0.015708577 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:39.357 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 podman[65354]: 2024-09-11 22:11:39.094092662 +0000 UTC m=+0.142314241 container create a592d442b3a63c3b979ea45a57f468b6954df1fb41769dbce2856fecf2033c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T22:11:39.405 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:39.789 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 podman[65354]: 2024-09-11 22:11:39.357499246 +0000 UTC m=+0.405720825 container init a592d442b3a63c3b979ea45a57f468b6954df1fb41769dbce2856fecf2033c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T22:11:39.789 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 podman[65354]: 2024-09-11 22:11:39.360514322 +0000 UTC m=+0.408735913 container start a592d442b3a63c3b979ea45a57f468b6954df1fb41769dbce2856fecf2033c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:11:39.789 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 ceph-mds[65373]: -- 172.21.15.153:0/3955857966 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eccee5b860 con 0x55eccee39400 2024-09-11T22:11:39.790 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c[65369]: starting mds.c at 2024-09-11T22:11:39.790 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 bash[65354]: a592d442b3a63c3b979ea45a57f468b6954df1fb41769dbce2856fecf2033c31 2024-09-11T22:11:39.790 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:11:39 smithi153 systemd[1]: Started Ceph mds.c for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 -- 172.21.15.92:0/816336868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798106a80 msgr2=0x7f8798106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 --2- 172.21.15.92:0/816336868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798106a80 0x7f8798106e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8788009930 tx=0x7f878802a420 comp rx=0 tx=0).stop 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 -- 172.21.15.92:0/816336868 shutdown_connections 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 --2- 172.21.15.92:0/816336868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8798101420 0x7f879810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 --2- 172.21.15.92:0/816336868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8798100a80 0x7f8798100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 --2- 172.21.15.92:0/816336868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798106a80 0x7f8798106e60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 -- 172.21.15.92:0/816336868 >> 172.21.15.92:0/816336868 conn(0x7f87980fc7a0 msgr2=0x7f87980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 -- 172.21.15.92:0/816336868 shutdown_connections 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.865+0000 7f879d306640 1 -- 172.21.15.92:0/816336868 wait complete. 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 Processor -- start 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 -- start start 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798100a80 0x7f879819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8798101420 0x7f879819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8798106a80 0x7f879819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8798110d50 con 0x7f8798100a80 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8798110bd0 con 0x7f8798106a80 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f879d306640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8798110ed0 con 0x7f8798101420 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798100a80 0x7f879819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798100a80 0x7f879819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44396/0 (socket says 172.21.15.92:44396) 2024-09-11T22:11:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 -- 172.21.15.92:0/640436458 learned_addr learned my addr 172.21.15.92:0/640436458 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f87977fe640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8798106a80 0x7f879819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f878e7fc640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8798101420 0x7f879819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 -- 172.21.15.92:0/640436458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8798101420 msgr2=0x7f879819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8798101420 0x7f879819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 -- 172.21.15.92:0/640436458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8798106a80 msgr2=0x7f879819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8798106a80 0x7f879819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.866+0000 7f8796ffd640 1 -- 172.21.15.92:0/640436458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879819f5e0 con 0x7f8798100a80 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.867+0000 7f8796ffd640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798100a80 0x7f879819a630 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8788005940 tx=0x7f87880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.867+0000 7f8794ff9640 1 -- 172.21.15.92:0/640436458 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878803f030 con 0x7f8798100a80 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.867+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879819f870 con 0x7f8798100a80 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.867+0000 7f8794ff9640 1 -- 172.21.15.92:0/640436458 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8788003cb0 con 0x7f8798100a80 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.867+0000 7f8794ff9640 1 -- 172.21.15.92:0/640436458 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788035070 con 0x7f8798100a80 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.867+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87981a71b0 con 0x7f8798100a80 2024-09-11T22:11:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.868+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8798100ee0 con 0x7f8798100a80 2024-09-11T22:11:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.869+0000 7f8794ff9640 1 -- 172.21.15.92:0/640436458 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8788042030 con 0x7f8798100a80 2024-09-11T22:11:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.869+0000 7f8794ff9640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87840a2c00 0x7f87840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.870+0000 7f878e7fc640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87840a2c00 0x7f87840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.870+0000 7f8794ff9640 1 -- 172.21.15.92:0/640436458 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f87880eeff0 con 0x7f8798100a80 2024-09-11T22:11:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.870+0000 7f878e7fc640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87840a2c00 0x7f87840a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87980febe0 tx=0x7f8780009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:39.874+0000 7f8794ff9640 1 -- 172.21.15.92:0/640436458 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87880bc3f0 con 0x7f8798100a80 2024-09-11T22:11:40.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.075+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f879819b990 con 0x7f8798100a80 2024-09-11T22:11:40.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.075+0000 7f8794ff9640 1 -- 172.21.15.92:0/640436458 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6394 (secure 0 0 0) 0x7f87880bbcf0 con 0x7f8798100a80 2024-09-11T22:11:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:40.076 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"btime":"2024-09-11T22:11:39:438781+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T22:11:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2024-09-11T22:11:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87840a2c00 msgr2=0x7f87840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87840a2c00 0x7f87840a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87980febe0 tx=0x7f8780009110 comp rx=0 tx=0).stop 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798100a80 msgr2=0x7f879819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798100a80 0x7f879819a630 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8788005940 tx=0x7f87880042f0 comp rx=0 tx=0).stop 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 shutdown_connections 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87840a2c00 0x7f87840a50c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8798106a80 0x7f879819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8798101420 0x7f879819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 --2- 172.21.15.92:0/640436458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798100a80 0x7f879819a630 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 >> 172.21.15.92:0/640436458 conn(0x7f87980fc7a0 msgr2=0x7f879806ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 shutdown_connections 2024-09-11T22:11:40.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:40.077+0000 7f879d306640 1 -- 172.21.15.92:0/640436458 wait complete. 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] up:boot 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: fsmap 7 up:standby 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: Deploying daemon mds.h on smithi110 2024-09-11T22:11:40.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:40.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:40 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/640436458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] up:boot 2024-09-11T22:11:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: fsmap 7 up:standby 2024-09-11T22:11:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:11:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:40.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:40.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:40.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: Deploying daemon mds.h on smithi110 2024-09-11T22:11:40.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:40.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:40 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/640436458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:40.788 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] up:boot 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: fsmap 7 up:standby 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: Deploying daemon mds.h on smithi110 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:40.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:40 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/640436458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:41.717 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:41 smithi110 systemd[1]: Starting Ceph mds.h for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:41.994 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:41 smithi110 podman[64829]: 2024-09-11 22:11:41.639473208 +0000 UTC m=+0.015541512 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:41.995 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:41 smithi110 podman[64829]: 2024-09-11 22:11:41.758904139 +0000 UTC m=+0.134972433 container create 40db97aa53028368176027065cfb826f829d646a20abbc15b25bcca9515472fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:11:42.467 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:41 smithi110 podman[64829]: 2024-09-11 22:11:41.994947183 +0000 UTC m=+0.371015475 container init 40db97aa53028368176027065cfb826f829d646a20abbc15b25bcca9515472fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T22:11:42.467 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:41 smithi110 podman[64829]: 2024-09-11 22:11:41.997724269 +0000 UTC m=+0.373792570 container start 40db97aa53028368176027065cfb826f829d646a20abbc15b25bcca9515472fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T22:11:42.467 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mds[64848]: -- 172.21.15.110:0/2802678843 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3e9d5f860 con 0x55d3e9d3d400 2024-09-11T22:11:42.467 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h[64844]: starting mds.h at 2024-09-11T22:11:42.467 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:42 smithi110 bash[64829]: 40db97aa53028368176027065cfb826f829d646a20abbc15b25bcca9515472fd 2024-09-11T22:11:42.467 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:11:42 smithi110 systemd[1]: Started Ceph mds.h for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:42.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:42.836 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:42 smithi153 ceph-mon[35045]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:42.836 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:42.836 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:42.836 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:42.837 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:42.837 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:42.837 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:42 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:42.972 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:42 smithi092 ceph-mon[31173]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:43.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:42 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mon[34830]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:42 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.543+0000 7f04f3105640 1 -- 172.21.15.92:0/644837224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec102db0 msgr2=0x7f04ec103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.543+0000 7f04f3105640 1 --2- 172.21.15.92:0/644837224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec102db0 0x7f04ec103190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009930 tx=0x7f04e002a420 comp rx=0 tx=0).stop 2024-09-11T22:11:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 -- 172.21.15.92:0/644837224 shutdown_connections 2024-09-11T22:11:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 --2- 172.21.15.92:0/644837224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04ec1082e0 0x7f04ec10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 --2- 172.21.15.92:0/644837224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04ec103760 0x7f04ec107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 --2- 172.21.15.92:0/644837224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec102db0 0x7f04ec103190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 -- 172.21.15.92:0/644837224 >> 172.21.15.92:0/644837224 conn(0x7f04ec0fc7d0 msgr2=0x7f04ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 -- 172.21.15.92:0/644837224 shutdown_connections 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 -- 172.21.15.92:0/644837224 wait complete. 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 Processor -- start 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.544+0000 7f04f3105640 1 -- start start 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f3105640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04ec102db0 0x7f04ec100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f3105640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec103760 0x7f04ec100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f3105640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04ec1082e0 0x7f04ec101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f3105640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec111f90 con 0x7f04ec103760 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f3105640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec111e10 con 0x7f04ec1082e0 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f3105640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec112110 con 0x7f04ec102db0 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec103760 0x7f04ec100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec103760 0x7f04ec100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44422/0 (socket says 172.21.15.92:44422) 2024-09-11T22:11:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 -- 172.21.15.92:0/3539623190 learned_addr learned my addr 172.21.15.92:0/3539623190 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f16a2640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04ec1082e0 0x7f04ec101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f0ea1640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04ec102db0 0x7f04ec100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 -- 172.21.15.92:0/3539623190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04ec102db0 msgr2=0x7f04ec100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04ec102db0 0x7f04ec100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 -- 172.21.15.92:0/3539623190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04ec1082e0 msgr2=0x7f04ec101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04ec1082e0 0x7f04ec101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 -- 172.21.15.92:0/3539623190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec1a69d0 con 0x7f04ec103760 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f0ea1640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04ec102db0 0x7f04ec100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04ebfff640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec103760 0x7f04ec100f50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04d800d950 tx=0x7f04d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04e9ffb640 1 -- 172.21.15.92:0/3539623190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8017070 con 0x7f04ec103760 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04e9ffb640 1 -- 172.21.15.92:0/3539623190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d8009c30 con 0x7f04ec103760 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ec1a6cc0 con 0x7f04ec103760 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.545+0000 7f04e9ffb640 1 -- 172.21.15.92:0/3539623190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d80047a0 con 0x7f04ec103760 2024-09-11T22:11:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.546+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ec1a71b0 con 0x7f04ec103760 2024-09-11T22:11:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.547+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ec103190 con 0x7f04ec103760 2024-09-11T22:11:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.547+0000 7f04e9ffb640 1 -- 172.21.15.92:0/3539623190 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f04d8004020 con 0x7f04ec103760 2024-09-11T22:11:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.548+0000 7f04e9ffb640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04dc0a2b80 0x7f04dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.548+0000 7f04e9ffb640 1 -- 172.21.15.92:0/3539623190 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f04d80ce8a0 con 0x7f04ec103760 2024-09-11T22:11:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.548+0000 7f04f0ea1640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04dc0a2b80 0x7f04dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.549+0000 7f04f0ea1640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04dc0a2b80 0x7f04dc0a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f04e0006010 tx=0x7f04e0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.552+0000 7f04e9ffb640 1 -- 172.21.15.92:0/3539623190 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04d8020020 con 0x7f04ec103760 2024-09-11T22:11:43.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.758+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04ec10a2c0 con 0x7f04ec103760 2024-09-11T22:11:43.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.759+0000 7f04e9ffb640 1 -- 172.21.15.92:0/3539623190 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7222 (secure 0 0 0) 0x7f04d809bc00 con 0x7f04ec103760 2024-09-11T22:11:43.759 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:43.759 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"btime":"2024-09-11T22:11:42:729074+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T22:11:43.759 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2024-09-11T22:11:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04dc0a2b80 msgr2=0x7f04dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04dc0a2b80 0x7f04dc0a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f04e0006010 tx=0x7f04e0005a20 comp rx=0 tx=0).stop 2024-09-11T22:11:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec103760 msgr2=0x7f04ec100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec103760 0x7f04ec100f50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04d800d950 tx=0x7f04d800de10 comp rx=0 tx=0).stop 2024-09-11T22:11:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 shutdown_connections 2024-09-11T22:11:43.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04dc0a2b80 0x7f04dc0a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04ec1082e0 0x7f04ec101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04ec103760 0x7f04ec100f50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 --2- 172.21.15.92:0/3539623190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04ec102db0 0x7f04ec100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:43.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 >> 172.21.15.92:0/3539623190 conn(0x7f04ec0fc7d0 msgr2=0x7f04ec105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:43.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 shutdown_connections 2024-09-11T22:11:43.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:43.761+0000 7f04f3105640 1 -- 172.21.15.92:0/3539623190 wait complete. 2024-09-11T22:11:43.863 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:43 smithi153 ceph-mon[35045]: Deploying daemon mds.i on smithi153 2024-09-11T22:11:43.863 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:43 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] up:boot 2024-09-11T22:11:43.863 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:43 smithi153 ceph-mon[35045]: fsmap 8 up:standby 2024-09-11T22:11:43.863 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:43 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:11:44.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:43 smithi092 ceph-mon[31173]: Deploying daemon mds.i on smithi153 2024-09-11T22:11:44.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:43 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] up:boot 2024-09-11T22:11:44.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:43 smithi092 ceph-mon[31173]: fsmap 8 up:standby 2024-09-11T22:11:44.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:43 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:11:44.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:43 smithi110 ceph-mon[34830]: Deploying daemon mds.i on smithi153 2024-09-11T22:11:44.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:43 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] up:boot 2024-09-11T22:11:44.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:43 smithi110 ceph-mon[34830]: fsmap 8 up:standby 2024-09-11T22:11:44.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:43 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:11:44.260 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 systemd[1]: Starting Ceph mds.i for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:44.522 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 podman[65658]: 2024-09-11 22:11:44.260144619 +0000 UTC m=+0.109723589 container create f9f398948c24a9a3834d3a74141bd9904a3fec02855f06a4bb812df3d9a77640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T22:11:44.523 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 podman[65658]: 2024-09-11 22:11:44.166405213 +0000 UTC m=+0.015984196 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:44.523 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 podman[65658]: 2024-09-11 22:11:44.496431706 +0000 UTC m=+0.346010685 container init f9f398948c24a9a3834d3a74141bd9904a3fec02855f06a4bb812df3d9a77640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:11:44.523 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 podman[65658]: 2024-09-11 22:11:44.499355114 +0000 UTC m=+0.348934086 container start f9f398948c24a9a3834d3a74141bd9904a3fec02855f06a4bb812df3d9a77640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3539623190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:44.789 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-mds[65678]: -- 172.21.15.153:0/2972296702 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570a3167860 con 0x5570a3144800 2024-09-11T22:11:44.790 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i[65674]: starting mds.i at 2024-09-11T22:11:44.790 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 bash[65658]: f9f398948c24a9a3834d3a74141bd9904a3fec02855f06a4bb812df3d9a77640 2024-09-11T22:11:44.790 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:11:44 smithi153 systemd[1]: Started Ceph mds.i for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3539623190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:45.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:45.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:44 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3539623190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:45.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:44 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:46.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:45 smithi092 ceph-mon[31173]: Deploying daemon mds.g on smithi092 2024-09-11T22:11:46.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:45 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] up:boot 2024-09-11T22:11:46.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:45 smithi092 ceph-mon[31173]: fsmap 9 up:standby 2024-09-11T22:11:46.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:45 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:11:46.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:45 smithi153 ceph-mon[35045]: Deploying daemon mds.g on smithi092 2024-09-11T22:11:46.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:45 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] up:boot 2024-09-11T22:11:46.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:45 smithi153 ceph-mon[35045]: fsmap 9 up:standby 2024-09-11T22:11:46.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:45 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:11:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:45 smithi110 ceph-mon[34830]: Deploying daemon mds.g on smithi092 2024-09-11T22:11:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:45 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] up:boot 2024-09-11T22:11:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:45 smithi110 ceph-mon[34830]: fsmap 9 up:standby 2024-09-11T22:11:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:45 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:11:46.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:46.739 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:46 smithi092 systemd[1]: Starting Ceph mds.g for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:47.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:46 smithi092 ceph-mon[31173]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:47.007 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:46 smithi092 podman[72535]: 2024-09-11 22:11:46.641193825 +0000 UTC m=+0.015765066 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:47.007 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:46 smithi092 podman[72535]: 2024-09-11 22:11:46.744153065 +0000 UTC m=+0.118724293 container create 8f461e252924867d6f65e742a4d7cfe48c06692688f3f792ebe9fda2dc7cfa99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T22:11:47.008 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:46 smithi092 podman[72535]: 2024-09-11 22:11:46.980627472 +0000 UTC m=+0.355198700 container init 8f461e252924867d6f65e742a4d7cfe48c06692688f3f792ebe9fda2dc7cfa99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T22:11:47.008 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:46 smithi092 podman[72535]: 2024-09-11 22:11:46.983915582 +0000 UTC m=+0.358486822 container start 8f461e252924867d6f65e742a4d7cfe48c06692688f3f792ebe9fda2dc7cfa99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T22:11:47.008 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:47 smithi092 ceph-mds[72578]: -- 172.21.15.92:0/3533823111 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56256be35860 con 0x56256be13400 2024-09-11T22:11:47.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:46 smithi153 ceph-mon[35045]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:46 smithi110 ceph-mon[34830]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:47.268 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:47 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g[72562]: starting mds.g at 2024-09-11T22:11:47.268 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:47 smithi092 bash[72535]: 8f461e252924867d6f65e742a4d7cfe48c06692688f3f792ebe9fda2dc7cfa99 2024-09-11T22:11:47.268 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:11:47 smithi092 systemd[1]: Started Ceph mds.g for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3865624468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 msgr2=0x7f8e84103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3865624468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e84103120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009ca0 tx=0x7f8e7c02a950 comp rx=0 tx=0).stop 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3865624468 shutdown_connections 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3865624468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8410c170 0x7f8e8410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3865624468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e84103660 0x7f8e8410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3865624468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e84103120 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3865624468 >> 172.21.15.92:0/3865624468 conn(0x7f8e840fc7a0 msgr2=0x7f8e840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3865624468 shutdown_connections 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.658+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3865624468 wait complete. 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 Processor -- start 2024-09-11T22:11:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 -- start start 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e8419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e84103660 0x7f8e8419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8410c170 0x7f8e8419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84101d70 con 0x7f8e84102d40 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84101bf0 con 0x7f8e8410c170 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e8ac46640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84101ef0 con 0x7f8e84103660 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e8419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e8419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42126/0 (socket says 172.21.15.92:42126) 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 -- 172.21.15.92:0/3351964378 learned_addr learned my addr 172.21.15.92:0/3351964378 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e891e3640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8410c170 0x7f8e8419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e83fff640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e84103660 0x7f8e8419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 -- 172.21.15.92:0/3351964378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e84103660 msgr2=0x7f8e8419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e84103660 0x7f8e8419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 -- 172.21.15.92:0/3351964378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8410c170 msgr2=0x7f8e8419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8410c170 0x7f8e8419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 -- 172.21.15.92:0/3351964378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8419f690 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.659+0000 7f8e889e2640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e8419a6e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005cc0 tx=0x7f8e7c02f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.660+0000 7f8e81ffb640 1 -- 172.21.15.92:0/3351964378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c034040 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.660+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8419f920 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.660+0000 7f8e81ffb640 1 -- 172.21.15.92:0/3351964378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e7c003e40 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.660+0000 7f8e81ffb640 1 -- 172.21.15.92:0/3351964378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c045480 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.660+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e841a7260 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.661+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e841031a0 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.662+0000 7f8e81ffb640 1 -- 172.21.15.92:0/3351964378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8e7c037020 con 0x7f8e84102d40 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.662+0000 7f8e81ffb640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e6c0a2b80 0x7f8e6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.663+0000 7f8e83fff640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e6c0a2b80 0x7f8e6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.663+0000 7f8e81ffb640 1 -- 172.21.15.92:0/3351964378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8e7c0f0000 con 0x7f8e84102d40 2024-09-11T22:11:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.663+0000 7f8e83fff640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e6c0a2b80 0x7f8e6c0a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e84100270 tx=0x7f8e78008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.666+0000 7f8e81ffb640 1 -- 172.21.15.92:0/3351964378 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e7c0811b0 con 0x7f8e84102d40 2024-09-11T22:11:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.888+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e841068f0 con 0x7f8e84102d40 2024-09-11T22:11:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.888+0000 7f8e81ffb640 1 -- 172.21.15.92:0/3351964378 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8875 (secure 0 0 0) 0x7f8e7c0bd400 con 0x7f8e84102d40 2024-09-11T22:11:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:47.889 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":11,"btime":"2024-09-11T22:11:47:751289+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T22:11:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 11 2024-09-11T22:11:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e6c0a2b80 msgr2=0x7f8e6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e6c0a2b80 0x7f8e6c0a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e84100270 tx=0x7f8e78008040 comp rx=0 tx=0).stop 2024-09-11T22:11:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 msgr2=0x7f8e8419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e8419a6e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005cc0 tx=0x7f8e7c02f650 comp rx=0 tx=0).stop 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 shutdown_connections 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e6c0a2b80 0x7f8e6c0a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8410c170 0x7f8e8419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e84103660 0x7f8e8419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 --2- 172.21.15.92:0/3351964378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e84102d40 0x7f8e8419a6e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 >> 172.21.15.92:0/3351964378 conn(0x7f8e840fc7a0 msgr2=0x7f8e84103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 shutdown_connections 2024-09-11T22:11:47.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:47.891+0000 7f8e8ac46640 1 -- 172.21.15.92:0/3351964378 wait complete. 2024-09-11T22:11:48.248 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: Deploying daemon mds.a on smithi092 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.92:6838/1790524059,v1:172.21.15.92:6839/1790524059] up:boot 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: fsmap 10 up:standby 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:11:48.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:48 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3351964378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: Deploying daemon mds.a on smithi092 2024-09-11T22:11:48.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:48.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.92:6838/1790524059,v1:172.21.15.92:6839/1790524059] up:boot 2024-09-11T22:11:48.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: fsmap 10 up:standby 2024-09-11T22:11:48.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:11:48.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:48 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3351964378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: Deploying daemon mds.a on smithi092 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.92:6838/1790524059,v1:172.21.15.92:6839/1790524059] up:boot 2024-09-11T22:11:48.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: fsmap 10 up:standby 2024-09-11T22:11:48.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:11:48.540 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:48 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3351964378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:50.018 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:49 smithi092 systemd[1]: Starting Ceph mds.a for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:50.320 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 podman[72930]: 2024-09-11 22:11:49.968242216 +0000 UTC m=+0.015990948 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:50.320 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 podman[72930]: 2024-09-11 22:11:50.084418518 +0000 UTC m=+0.132167250 container create 77b3d4333b9def4ec8aae755c8798b262e4e90bcfb494dcb9ccb0bae7e72a66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T22:11:50.320 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 podman[72930]: 2024-09-11 22:11:50.320410616 +0000 UTC m=+0.368159348 container init 77b3d4333b9def4ec8aae755c8798b262e4e90bcfb494dcb9ccb0bae7e72a66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T22:11:50.739 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 podman[72930]: 2024-09-11 22:11:50.323326314 +0000 UTC m=+0.371075049 container start 77b3d4333b9def4ec8aae755c8798b262e4e90bcfb494dcb9ccb0bae7e72a66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T22:11:50.739 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mds[72949]: -- 172.21.15.92:0/543827530 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a58594b860 con 0x55a585929400 2024-09-11T22:11:50.739 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a[72945]: starting mds.a at 2024-09-11T22:11:50.739 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 bash[72930]: 77b3d4333b9def4ec8aae755c8798b262e4e90bcfb494dcb9ccb0bae7e72a66e 2024-09-11T22:11:50.740 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:11:50 smithi092 systemd[1]: Started Ceph mds.a for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:50.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:51.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mon[31173]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:51.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:51.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:51.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:50 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:50 smithi153 ceph-mon[35045]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:50 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:51.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:50 smithi110 ceph-mon[34830]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:51.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:51.054 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:11:51.054 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:11:51.054 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:50 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:51.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 -- 172.21.15.92:0/2545245468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 msgr2=0x7f16f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:51.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 --2- 172.21.15.92:0/2545245468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f01029b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009900 tx=0x7f16e402a480 comp rx=0 tx=0).stop 2024-09-11T22:11:51.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 -- 172.21.15.92:0/2545245468 shutdown_connections 2024-09-11T22:11:51.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 --2- 172.21.15.92:0/2545245468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16f0102ef0 0x7f16f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 --2- 172.21.15.92:0/2545245468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f01029b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 --2- 172.21.15.92:0/2545245468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f0100e20 0x7f16f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 -- 172.21.15.92:0/2545245468 >> 172.21.15.92:0/2545245468 conn(0x7f16f00fc7d0 msgr2=0x7f16f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 -- 172.21.15.92:0/2545245468 shutdown_connections 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.731+0000 7f16f5d00640 1 -- 172.21.15.92:0/2545245468 wait complete. 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 Processor -- start 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 -- start start 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16f0100e20 0x7f16f01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f0197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f0102ef0 0x7f16f019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0110e50 con 0x7f16f0102570 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0110cd0 con 0x7f16f0102ef0 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16f5d00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0110fd0 con 0x7f16f0100e20 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16eeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f0197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:51.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16eeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f0197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42160/0 (socket says 172.21.15.92:42160) 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16eeffd640 1 -- 172.21.15.92:0/1817467531 learned_addr learned my addr 172.21.15.92:0/1817467531 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.732+0000 7f16ef7fe640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16f0100e20 0x7f16f01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16effff640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f0102ef0 0x7f16f019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16eeffd640 1 -- 172.21.15.92:0/1817467531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16f0100e20 msgr2=0x7f16f01972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16eeffd640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16f0100e20 0x7f16f01972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16eeffd640 1 -- 172.21.15.92:0/1817467531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f0102ef0 msgr2=0x7f16f019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16eeffd640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f0102ef0 0x7f16f019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16eeffd640 1 -- 172.21.15.92:0/1817467531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f019c280 con 0x7f16f0102570 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16effff640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f0102ef0 0x7f16f019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16eeffd640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f0197810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16ecff9640 1 -- 172.21.15.92:0/1817467531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e403f070 con 0x7f16f0102570 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16ecff9640 1 -- 172.21.15.92:0/1817467531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16e4003aa0 con 0x7f16f0102570 2024-09-11T22:11:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f019c510 con 0x7f16f0102570 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16ecff9640 1 -- 172.21.15.92:0/1817467531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4035030 con 0x7f16f0102570 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.733+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f01a3e50 con 0x7f16f0102570 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.734+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f01021b0 con 0x7f16f0102570 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.736+0000 7f16ecff9640 1 -- 172.21.15.92:0/1817467531 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f16e4042070 con 0x7f16f0102570 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.736+0000 7f16ecff9640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f16d80a2b80 0x7f16d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.737+0000 7f16ef7fe640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f16d80a2b80 0x7f16d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.737+0000 7f16ecff9640 1 -- 172.21.15.92:0/1817467531 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f16e40efbc0 con 0x7f16f0102570 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.737+0000 7f16ef7fe640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f16d80a2b80 0x7f16d80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16e000abe0 tx=0x7f16e000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.740+0000 7f16ecff9640 1 -- 172.21.15.92:0/1817467531 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16e40bcfc0 con 0x7f16f0102570 2024-09-11T22:11:51.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.940+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16f0101200 con 0x7f16f0102570 2024-09-11T22:11:51.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:51 smithi092 ceph-mon[31173]: Deploying daemon mds.b on smithi110 2024-09-11T22:11:51.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:51 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.92:6840/2477024139,v1:172.21.15.92:6841/2477024139] up:boot 2024-09-11T22:11:51.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:51 smithi092 ceph-mon[31173]: fsmap 11 up:standby 2024-09-11T22:11:51.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:51 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:11:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.940+0000 7f16ecff9640 1 -- 172.21.15.92:0/1817467531 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9701 (secure 0 0 0) 0x7f16f0101200 con 0x7f16f0102570 2024-09-11T22:11:51.941 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:51.941 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"btime":"2024-09-11T22:11:50:730391+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T22:11:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2024-09-11T22:11:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f16d80a2b80 msgr2=0x7f16d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f16d80a2b80 0x7f16d80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16e000abe0 tx=0x7f16e000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 msgr2=0x7f16f0197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f0197810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e40042f0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 shutdown_connections 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f16d80a2b80 0x7f16d80a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f0102ef0 0x7f16f019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f0102570 0x7f16f0197810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 --2- 172.21.15.92:0/1817467531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16f0100e20 0x7f16f01972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.942+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 >> 172.21.15.92:0/1817467531 conn(0x7f16f00fc7d0 msgr2=0x7f16f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.943+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 shutdown_connections 2024-09-11T22:11:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:51.943+0000 7f16f5d00640 1 -- 172.21.15.92:0/1817467531 wait complete. 2024-09-11T22:11:52.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:51 smithi153 ceph-mon[35045]: Deploying daemon mds.b on smithi110 2024-09-11T22:11:52.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:51 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.92:6840/2477024139,v1:172.21.15.92:6841/2477024139] up:boot 2024-09-11T22:11:52.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:51 smithi153 ceph-mon[35045]: fsmap 11 up:standby 2024-09-11T22:11:52.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:51 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:11:52.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:51 smithi110 ceph-mon[34830]: Deploying daemon mds.b on smithi110 2024-09-11T22:11:52.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:51 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.92:6840/2477024139,v1:172.21.15.92:6841/2477024139] up:boot 2024-09-11T22:11:52.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:51 smithi110 ceph-mon[34830]: fsmap 11 up:standby 2024-09-11T22:11:52.053 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:51 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:11:52.440 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:52 smithi110 systemd[1]: Starting Ceph mds.b for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:11:52.717 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:52 smithi110 podman[65131]: 2024-09-11 22:11:52.554764804 +0000 UTC m=+0.015615199 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:11:52.717 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:52 smithi110 podman[65131]: 2024-09-11 22:11:52.681466646 +0000 UTC m=+0.142317032 container create 94775968d4fb1ca341cef2b01b806bd120bc54e89b447b635b2bd1294a04b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T22:11:53.017 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:52 smithi110 ceph-mon[34830]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:53.017 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:52 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1817467531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:53.018 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:52 smithi110 podman[65131]: 2024-09-11 22:11:52.925742079 +0000 UTC m=+0.386592466 container init 94775968d4fb1ca341cef2b01b806bd120bc54e89b447b635b2bd1294a04b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T22:11:53.018 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:52 smithi110 podman[65131]: 2024-09-11 22:11:52.928698567 +0000 UTC m=+0.389548955 container start 94775968d4fb1ca341cef2b01b806bd120bc54e89b447b635b2bd1294a04b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T22:11:53.018 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:52 smithi110 ceph-mds[65150]: -- 172.21.15.110:0/1283643163 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6352e5860 con 0x55a6352c2800 2024-09-11T22:11:53.018 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:52 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: starting mds.b at 2024-09-11T22:11:53.018 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:53 smithi110 bash[65131]: 94775968d4fb1ca341cef2b01b806bd120bc54e89b447b635b2bd1294a04b9ee 2024-09-11T22:11:53.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:52 smithi092 ceph-mon[31173]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:53.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:52 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1817467531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:53.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:52 smithi153 ceph-mon[35045]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:53.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:52 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1817467531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:53.466 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:11:53 smithi110 systemd[1]: Started Ceph mds.b for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:54.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] up:boot 2024-09-11T22:11:54.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: fsmap 12 up:standby 2024-09-11T22:11:54.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:54 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:11:54.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] up:boot 2024-09-11T22:11:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: fsmap 12 up:standby 2024-09-11T22:11:54.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:54 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] up:boot 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: fsmap 12 up:standby 2024-09-11T22:11:54.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:54 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:11:54.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:11:54.860 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:55.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.404+0000 7fa4b3d77640 1 -- 172.21.15.92:0/1921565784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102570 msgr2=0x7fa4ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:55.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.404+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/1921565784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102570 0x7fa4ac1029b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009900 tx=0x7fa49c02a480 comp rx=0 tx=0).stop 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.404+0000 7fa4b3d77640 1 -- 172.21.15.92:0/1921565784 shutdown_connections 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.404+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/1921565784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4ac102ef0 0x7fa4ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.404+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/1921565784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102570 0x7fa4ac1029b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.404+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/1921565784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4ac100e20 0x7fa4ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.404+0000 7fa4b3d77640 1 -- 172.21.15.92:0/1921565784 >> 172.21.15.92:0/1921565784 conn(0x7fa4ac0fc7d0 msgr2=0x7fa4ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 -- 172.21.15.92:0/1921565784 shutdown_connections 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 -- 172.21.15.92:0/1921565784 wait complete. 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 Processor -- start 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 -- start start 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4ac100e20 0x7fa4ac198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4ac102570 0x7fa4ac198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102ef0 0x7fa4ac19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac110e30 con 0x7fa4ac102ef0 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac110cb0 con 0x7fa4ac102570 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.405+0000 7fa4b3d77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac110fb0 con 0x7fa4ac100e20 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102ef0 0x7fa4ac19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102ef0 0x7fa4ac19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50170/0 (socket says 172.21.15.92:50170) 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 -- 172.21.15.92:0/9001816 learned_addr learned my addr 172.21.15.92:0/9001816 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b1b13640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4ac100e20 0x7fa4ac198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b1312640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4ac102570 0x7fa4ac198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 -- 172.21.15.92:0/9001816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4ac100e20 msgr2=0x7fa4ac198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4ac100e20 0x7fa4ac198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 -- 172.21.15.92:0/9001816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4ac102570 msgr2=0x7fa4ac198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4ac102570 0x7fa4ac198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 -- 172.21.15.92:0/9001816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac19d3d0 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b2314640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102ef0 0x7fa4ac19ccf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400eb10 tx=0x7fa4a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4a2ffd640 1 -- 172.21.15.92:0/9001816 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4004b20 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4a2ffd640 1 -- 172.21.15.92:0/9001816 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4a4004cc0 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac19d6c0 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4a2ffd640 1 -- 172.21.15.92:0/9001816 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4010690 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.406+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ac1a4fa0 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.407+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ac101200 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.408+0000 7fa4a2ffd640 1 -- 172.21.15.92:0/9001816 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa4a4005020 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.409+0000 7fa4a2ffd640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa4980a2b80 0x7fa4980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.409+0000 7fa4b1b13640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa4980a2b80 0x7fa4980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.409+0000 7fa4a2ffd640 1 -- 172.21.15.92:0/9001816 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa4a40cf260 con 0x7fa4ac102ef0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.410+0000 7fa4b1b13640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa4980a2b80 0x7fa4980a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009bd0 tx=0x7fa4a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.413+0000 7fa4a2ffd640 1 -- 172.21.15.92:0/9001816 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4a409c660 con 0x7fa4ac102ef0 2024-09-11T22:11:55.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.629+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4ac10a0a0 con 0x7fa4ac102ef0 2024-09-11T22:11:55.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.630+0000 7fa4a2ffd640 1 -- 172.21.15.92:0/9001816 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10527 (secure 0 0 0) 0x7fa4a409bf60 con 0x7fa4ac102ef0 2024-09-11T22:11:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:55.630 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":13,"btime":"2024-09-11T22:11:53:749113+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T22:11:55.631 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 13 2024-09-11T22:11:55.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa4980a2b80 msgr2=0x7fa4980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:55.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa4980a2b80 0x7fa4980a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009bd0 tx=0x7fa4a8005f90 comp rx=0 tx=0).stop 2024-09-11T22:11:55.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102ef0 msgr2=0x7fa4ac19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:55.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102ef0 0x7fa4ac19ccf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400eb10 tx=0x7fa4a400c2d0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 shutdown_connections 2024-09-11T22:11:55.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa4980a2b80 0x7fa4980a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac102ef0 0x7fa4ac19ccf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4ac102570 0x7fa4ac198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 --2- 172.21.15.92:0/9001816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4ac100e20 0x7fa4ac198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:55.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 >> 172.21.15.92:0/9001816 conn(0x7fa4ac0fc7d0 msgr2=0x7fa4ac06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:55.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 shutdown_connections 2024-09-11T22:11:55.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:55.632+0000 7fa4b3d77640 1 -- 172.21.15.92:0/9001816 wait complete. 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:55.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:55.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:55 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:55.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.915 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:55.915 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:55.915 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.915 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:55 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:55.952 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:55.953 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:55 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:11:56.857 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:56 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/9001816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:56.857 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:56 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:56.857 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:56 smithi092 ceph-mon[31173]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:56.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T22:11:56.954 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:56 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/9001816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:56.954 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:56 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:56.954 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:56 smithi153 ceph-mon[35045]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:56.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:56 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/9001816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:11:56.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:56 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:56.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:56 smithi110 ceph-mon[34830]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:57.157 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:57.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 -- 172.21.15.92:0/326245670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102ef0 msgr2=0x7f274810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:57.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 --2- 172.21.15.92:0/326245670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102ef0 0x7f274810da40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2740009ca0 tx=0x7f274002a950 comp rx=0 tx=0).stop 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 -- 172.21.15.92:0/326245670 shutdown_connections 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 --2- 172.21.15.92:0/326245670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102ef0 0x7f274810da40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 --2- 172.21.15.92:0/326245670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2748102570 0x7f27481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 --2- 172.21.15.92:0/326245670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2748100e20 0x7f2748101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 -- 172.21.15.92:0/326245670 >> 172.21.15.92:0/326245670 conn(0x7f27480fc7d0 msgr2=0x7f27480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 -- 172.21.15.92:0/326245670 shutdown_connections 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 -- 172.21.15.92:0/326245670 wait complete. 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.642+0000 7f274ec10640 1 Processor -- start 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274ec10640 1 -- start start 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274ec10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2748100e20 0x7f27481970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274ec10640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102570 0x7f27481975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274ec10640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2748102ef0 0x7f2748197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274ec10640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2748110e30 con 0x7f2748102570 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274ec10640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2748110cb0 con 0x7f2748100e20 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274ec10640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2748110fb0 con 0x7f2748102ef0 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102570 0x7f27481975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102570 0x7f27481975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50184/0 (socket says 172.21.15.92:50184) 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 -- 172.21.15.92:0/704951286 learned_addr learned my addr 172.21.15.92:0/704951286 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274c9ac640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2748100e20 0x7f27481970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274d1ad640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2748102ef0 0x7f2748197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 -- 172.21.15.92:0/704951286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2748102ef0 msgr2=0x7f2748197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2748102ef0 0x7f2748197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 -- 172.21.15.92:0/704951286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2748100e20 msgr2=0x7f27481970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2748100e20 0x7f27481970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 -- 172.21.15.92:0/704951286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27481a6920 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f274d1ad640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2748102ef0 0x7f2748197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.643+0000 7f2747fff640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102570 0x7f27481975f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f273400c890 tx=0x7f273400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.644+0000 7f2745ffb640 1 -- 172.21.15.92:0/704951286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734004220 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.644+0000 7f2745ffb640 1 -- 172.21.15.92:0/704951286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27340043c0 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.644+0000 7f2745ffb640 1 -- 172.21.15.92:0/704951286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734004a00 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.644+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27481a6c10 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.644+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27481a71a0 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.645+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2748101200 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.646+0000 7f2745ffb640 1 -- 172.21.15.92:0/704951286 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2734004ba0 con 0x7f2748102570 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.646+0000 7f2745ffb640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27380a2c00 0x7f27380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.646+0000 7f274c9ac640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27380a2c00 0x7f27380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:57.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.646+0000 7f2745ffb640 1 -- 172.21.15.92:0/704951286 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f27340ce6c0 con 0x7f2748102570 2024-09-11T22:11:57.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.647+0000 7f274c9ac640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27380a2c00 0x7f27380a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f273c009bd0 tx=0x7f273c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:57.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.651+0000 7f2745ffb640 1 -- 172.21.15.92:0/704951286 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2734014030 con 0x7f2748102570 2024-09-11T22:11:57.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:57 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.835+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2748109090 con 0x7f2748102570 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.836+0000 7f2745ffb640 1 -- 172.21.15.92:0/704951286 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f273409b470 con 0x7f2748102570 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:11:57.837 INFO:teuthology.orchestra.run.smithi092.stdout:] 2024-09-11T22:11:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27380a2c00 msgr2=0x7f27380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27380a2c00 0x7f27380a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f273c009bd0 tx=0x7f273c005f90 comp rx=0 tx=0).stop 2024-09-11T22:11:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102570 msgr2=0x7f27481975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102570 0x7f27481975f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f273400c890 tx=0x7f273400cd50 comp rx=0 tx=0).stop 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 shutdown_connections 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27380a2c00 0x7f27380a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2748102ef0 0x7f2748197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2748102570 0x7f27481975f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 --2- 172.21.15.92:0/704951286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2748100e20 0x7f27481970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 >> 172.21.15.92:0/704951286 conn(0x7f27480fc7d0 msgr2=0x7f27480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 shutdown_connections 2024-09-11T22:11:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:57.838+0000 7f274ec10640 1 -- 172.21.15.92:0/704951286 wait complete. 2024-09-11T22:11:57.868 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.869 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.869 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.869 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:57 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.926 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.926 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.926 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:57.926 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:57 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:58.484 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T22:11:58.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T22:11:58.728 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:11:58.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:58 smithi092 ceph-mon[31173]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:58.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:58.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:58.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:58.986 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:58.986 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:58.986 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:58 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/704951286' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:11:59.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:58 smithi153 ceph-mon[35045]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:58 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/704951286' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:11:59.139 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:58 smithi110 ceph-mon[34830]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:11:59.140 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:59.140 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:59.140 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:11:59.140 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:11:59.140 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:11:59.140 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:58 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/704951286' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:11:59.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 -- 172.21.15.92:0/2597377166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 msgr2=0x7fb87010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:59.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 --2- 172.21.15.92:0/2597377166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87010da40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb868009ca0 tx=0x7fb86802a950 comp rx=0 tx=0).stop 2024-09-11T22:11:59.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 -- 172.21.15.92:0/2597377166 shutdown_connections 2024-09-11T22:11:59.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 --2- 172.21.15.92:0/2597377166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87010da40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 --2- 172.21.15.92:0/2597377166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb870102570 0x7fb8701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 --2- 172.21.15.92:0/2597377166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb870100e20 0x7fb870101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 -- 172.21.15.92:0/2597377166 >> 172.21.15.92:0/2597377166 conn(0x7fb8700fc7d0 msgr2=0x7fb8700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 -- 172.21.15.92:0/2597377166 shutdown_connections 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 -- 172.21.15.92:0/2597377166 wait complete. 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 Processor -- start 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.235+0000 7fb876d3e640 1 -- start start 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb876d3e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb870100e20 0x7fb870198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb876d3e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb870102570 0x7fb870198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb876d3e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb876d3e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb870110e30 con 0x7fb870102ef0 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb876d3e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb870110cb0 con 0x7fb870100e20 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb876d3e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb870110fb0 con 0x7fb870102570 2024-09-11T22:11:59.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50202/0 (socket says 172.21.15.92:50202) 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 -- 172.21.15.92:0/237589732 learned_addr learned my addr 172.21.15.92:0/237589732 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb874ada640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb870100e20 0x7fb870198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb86ffff640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb870102570 0x7fb870198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 -- 172.21.15.92:0/237589732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb870102570 msgr2=0x7fb870198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb870102570 0x7fb870198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 -- 172.21.15.92:0/237589732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb870100e20 msgr2=0x7fb870198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb870100e20 0x7fb870198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 -- 172.21.15.92:0/237589732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87019d2f0 con 0x7fb870102ef0 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb874ada640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb870100e20 0x7fb870198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb8752db640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87019cc10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb868004af0 tx=0x7fb86802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb86dffb640 1 -- 172.21.15.92:0/237589732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868034040 con 0x7fb870102ef0 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87019d580 con 0x7fb870102ef0 2024-09-11T22:11:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb86dffb640 1 -- 172.21.15.92:0/237589732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8680308f0 con 0x7fb870102ef0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.236+0000 7fb86dffb640 1 -- 172.21.15.92:0/237589732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868004670 con 0x7fb870102ef0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.237+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701a4ec0 con 0x7fb870102ef0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.238+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8701021b0 con 0x7fb870102ef0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.239+0000 7fb86dffb640 1 -- 172.21.15.92:0/237589732 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb868037020 con 0x7fb870102ef0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.239+0000 7fb86dffb640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8580a2b80 0x7fb8580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.240+0000 7fb874ada640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8580a2b80 0x7fb8580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.240+0000 7fb86dffb640 1 -- 172.21.15.92:0/237589732 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fb8680efc10 con 0x7fb870102ef0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.240+0000 7fb874ada640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8580a2b80 0x7fb8580a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb86000bfd0 tx=0x7fb86000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:11:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.243+0000 7fb86dffb640 1 -- 172.21.15.92:0/237589732 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8680811b0 con 0x7fb870102ef0 2024-09-11T22:11:59.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.410+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb870101200 con 0x7fb870102ef0 2024-09-11T22:11:59.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.730+0000 7fb86dffb640 1 -- 172.21.15.92:0/237589732 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7fb870101200 con 0x7fb870102ef0 2024-09-11T22:11:59.734 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2024-09-11T22:11:59.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8580a2b80 msgr2=0x7fb8580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:59.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8580a2b80 0x7fb8580a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb86000bfd0 tx=0x7fb86000b390 comp rx=0 tx=0).stop 2024-09-11T22:11:59.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 msgr2=0x7fb87019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:11:59.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87019cc10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb868004af0 tx=0x7fb86802f5d0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 shutdown_connections 2024-09-11T22:11:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8580a2b80 0x7fb8580a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb870102ef0 0x7fb87019cc10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb870102570 0x7fb870198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 --2- 172.21.15.92:0/237589732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb870100e20 0x7fb870198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:11:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 >> 172.21.15.92:0/237589732 conn(0x7fb8700fc7d0 msgr2=0x7fb87006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:11:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 shutdown_connections 2024-09-11T22:11:59.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:11:59.732+0000 7fb876d3e640 1 -- 172.21.15.92:0/237589732 wait complete. 2024-09-11T22:12:00.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:11:59 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/237589732' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:12:00.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:11:59 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/237589732' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:12:00.096 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:11:59 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/237589732' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:12:00.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-11T22:12:00.638 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:00 smithi092 ceph-mon[31173]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:00 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/237589732' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:12:01.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:00 smithi092 ceph-mon[31173]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:12:01.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:00 smithi153 ceph-mon[35045]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:01.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:00 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/237589732' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:12:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:00 smithi153 ceph-mon[35045]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.148+0000 7f395331d640 1 -- 172.21.15.92:0/2229772379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c100e20 msgr2=0x7f394c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 --2- 172.21.15.92:0/2229772379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c100e20 0x7f394c101200 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3940009900 tx=0x7f394002a480 comp rx=0 tx=0).stop 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 -- 172.21.15.92:0/2229772379 shutdown_connections 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 --2- 172.21.15.92:0/2229772379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c102ef0 0x7f394c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 --2- 172.21.15.92:0/2229772379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f394c102570 0x7f394c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 --2- 172.21.15.92:0/2229772379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c100e20 0x7f394c101200 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 -- 172.21.15.92:0/2229772379 >> 172.21.15.92:0/2229772379 conn(0x7f394c0fc7d0 msgr2=0x7f394c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 -- 172.21.15.92:0/2229772379 shutdown_connections 2024-09-11T22:12:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 -- 172.21.15.92:0/2229772379 wait complete. 2024-09-11T22:12:01.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 Processor -- start 2024-09-11T22:12:01.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.149+0000 7f395331d640 1 -- start start 2024-09-11T22:12:01.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f395331d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f394c100e20 0x7f394c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f395331d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c102570 0x7f394c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f395331d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c102ef0 0x7f394c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f395331d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f394c110ed0 con 0x7f394c102570 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f395331d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f394c110d50 con 0x7f394c100e20 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f395331d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f394c111050 con 0x7f394c102ef0 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c102570 0x7f394c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c102570 0x7f394c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50216/0 (socket says 172.21.15.92:50216) 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 -- 172.21.15.92:0/595725349 learned_addr learned my addr 172.21.15.92:0/595725349 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39510b9640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f394c100e20 0x7f394c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39518ba640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c102ef0 0x7f394c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 -- 172.21.15.92:0/595725349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c102ef0 msgr2=0x7f394c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c102ef0 0x7f394c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 -- 172.21.15.92:0/595725349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f394c100e20 msgr2=0x7f394c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f394c100e20 0x7f394c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 -- 172.21.15.92:0/595725349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c1a6a10 con 0x7f394c102570 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39518ba640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c102ef0 0x7f394c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f39508b8640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c102570 0x7f394c0fd1c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f394800b4a0 tx=0x7f394800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f393e7fc640 1 -- 172.21.15.92:0/595725349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948014070 con 0x7f394c102570 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c1a6d00 con 0x7f394c102570 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f393e7fc640 1 -- 172.21.15.92:0/595725349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3948010ac0 con 0x7f394c102570 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.150+0000 7f393e7fc640 1 -- 172.21.15.92:0/595725349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948017c30 con 0x7f394c102570 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.151+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c1a71e0 con 0x7f394c102570 2024-09-11T22:12:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.151+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c101200 con 0x7f394c102570 2024-09-11T22:12:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.153+0000 7f393e7fc640 1 -- 172.21.15.92:0/595725349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3948017070 con 0x7f394c102570 2024-09-11T22:12:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.153+0000 7f393e7fc640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f39380a2b80 0x7f39380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.153+0000 7f39510b9640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f39380a2b80 0x7f39380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.153+0000 7f393e7fc640 1 -- 172.21.15.92:0/595725349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7f39480ce3d0 con 0x7f394c102570 2024-09-11T22:12:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.154+0000 7f39510b9640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f39380a2b80 0x7f39380a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3940009230 tx=0x7f39400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:01.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.157+0000 7f393e7fc640 1 -- 172.21.15.92:0/595725349 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f394809b660 con 0x7f394c102570 2024-09-11T22:12:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:00 smithi110 ceph-mon[34830]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:00 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/237589732' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:12:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:00 smithi110 ceph-mon[34830]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:12:01.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.324+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f394c10a0a0 con 0x7f394c102570 2024-09-11T22:12:01.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.744+0000 7f393e7fc640 1 -- 172.21.15.92:0/595725349 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) ==== 171+0+0 (secure 0 0 0) 0x7f394809af60 con 0x7f394c102570 2024-09-11T22:12:01.745 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2024-09-11T22:12:01.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f39380a2b80 msgr2=0x7f39380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:01.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f39380a2b80 0x7f39380a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3940009230 tx=0x7f39400058d0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c102570 msgr2=0x7f394c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c102570 0x7f394c0fd1c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f394800b4a0 tx=0x7f394800b960 comp rx=0 tx=0).stop 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 shutdown_connections 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f39380a2b80 0x7f39380a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f394c102ef0 0x7f394c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394c102570 0x7f394c0fd1c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 --2- 172.21.15.92:0/595725349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f394c100e20 0x7f394c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 >> 172.21.15.92:0/595725349 conn(0x7f394c0fc7d0 msgr2=0x7f394c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 shutdown_connections 2024-09-11T22:12:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:01.746+0000 7f395331d640 1 -- 172.21.15.92:0/595725349 wait complete. 2024-09-11T22:12:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:01 smithi092 ceph-mon[31173]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:12:02.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:01 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/595725349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-11T22:12:02.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:01 smithi153 ceph-mon[35045]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:12:02.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:01 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/595725349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-11T22:12:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:01 smithi110 ceph-mon[34830]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:12:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:01 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/595725349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-11T22:12:02.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T22:12:02.670 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:02 smithi092 ceph-mon[31173]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:02 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/595725349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T22:12:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:02 smithi092 ceph-mon[31173]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:12:03.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:02 smithi092 ceph-mon[31173]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:12:03.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:02 smithi153 ceph-mon[35045]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:02 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/595725349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T22:12:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:02 smithi153 ceph-mon[35045]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:12:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:02 smithi153 ceph-mon[35045]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 -- 172.21.15.92:0/837296120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90102d40 msgr2=0x7faa90103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 --2- 172.21.15.92:0/837296120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90102d40 0x7faa90103120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa80009900 tx=0x7faa8002a480 comp rx=0 tx=0).stop 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 -- 172.21.15.92:0/837296120 shutdown_connections 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 --2- 172.21.15.92:0/837296120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa9010c170 0x7faa9010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 --2- 172.21.15.92:0/837296120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa90103660 0x7faa9010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 --2- 172.21.15.92:0/837296120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90102d40 0x7faa90103120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 -- 172.21.15.92:0/837296120 >> 172.21.15.92:0/837296120 conn(0x7faa900fc7a0 msgr2=0x7faa900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 -- 172.21.15.92:0/837296120 shutdown_connections 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.208+0000 7faa964ec640 1 -- 172.21.15.92:0/837296120 wait complete. 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 Processor -- start 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 -- start start 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa90102d40 0x7faa901984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90103660 0x7faa90198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9010c170 0x7faa9019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa90111ee0 con 0x7faa90103660 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faa90111d60 con 0x7faa90102d40 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.209+0000 7faa964ec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa90112060 con 0x7faa9010c170 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90103660 0x7faa90198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90103660 0x7faa90198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50236/0 (socket says 172.21.15.92:50236) 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 -- 172.21.15.92:0/2702469109 learned_addr learned my addr 172.21.15.92:0/2702469109 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 -- 172.21.15.92:0/2702469109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9010c170 msgr2=0x7faa9019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa94a89640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9010c170 0x7faa9019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8ffff640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa90102d40 0x7faa901984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9010c170 0x7faa9019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 -- 172.21.15.92:0/2702469109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa90102d40 msgr2=0x7faa901984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa90102d40 0x7faa901984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 -- 172.21.15.92:0/2702469109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9019d470 con 0x7faa90103660 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa94a89640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9010c170 0x7faa9019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8f7fe640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90103660 0x7faa90198a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa8400b3f0 tx=0x7faa8400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8d7fa640 1 -- 172.21.15.92:0/2702469109 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84015020 con 0x7faa90103660 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9019d760 con 0x7faa90103660 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8d7fa640 1 -- 172.21.15.92:0/2702469109 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa84011ce0 con 0x7faa90103660 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa8d7fa640 1 -- 172.21.15.92:0/2702469109 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84010d80 con 0x7faa90103660 2024-09-11T22:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.210+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a5040 con 0x7faa90103660 2024-09-11T22:12:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.211+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa901031a0 con 0x7faa90103660 2024-09-11T22:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.212+0000 7faa8d7fa640 1 -- 172.21.15.92:0/2702469109 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa84018020 con 0x7faa90103660 2024-09-11T22:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.213+0000 7faa8d7fa640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faa780a2b80 0x7faa780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.213+0000 7faa8ffff640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faa780a2b80 0x7faa780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.213+0000 7faa8d7fa640 1 -- 172.21.15.92:0/2702469109 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7faa840ce7e0 con 0x7faa90103660 2024-09-11T22:12:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.214+0000 7faa8ffff640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faa780a2b80 0x7faa780a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:02 smithi110 ceph-mon[34830]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:02 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/595725349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T22:12:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:02 smithi110 ceph-mon[34830]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:12:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:02 smithi110 ceph-mon[34830]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:12:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.217+0000 7faa8d7fa640 1 -- 172.21.15.92:0/2702469109 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa8401f020 con 0x7faa90103660 2024-09-11T22:12:03.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.418+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faa9002d7c0 con 0x7faa90103660 2024-09-11T22:12:03.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.763+0000 7faa8d7fa640 1 -- 172.21.15.92:0/2702469109 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7faa8409b8f0 con 0x7faa90103660 2024-09-11T22:12:03.764 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T22:12:03.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faa780a2b80 msgr2=0x7faa780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:03.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faa780a2b80 0x7faa780a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa80000bc0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90103660 msgr2=0x7faa90198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:03.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90103660 0x7faa90198a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa8400b3f0 tx=0x7faa8400b8b0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 shutdown_connections 2024-09-11T22:12:03.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7faa780a2b80 0x7faa780a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa9010c170 0x7faa9019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa90103660 0x7faa90198a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 --2- 172.21.15.92:0/2702469109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa90102d40 0x7faa901984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:03.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 >> 172.21.15.92:0/2702469109 conn(0x7faa900fc7a0 msgr2=0x7faa900fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:03.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 shutdown_connections 2024-09-11T22:12:03.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:03.765+0000 7faa964ec640 1 -- 172.21.15.92:0/2702469109 wait complete. 2024-09-11T22:12:04.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:03 smithi092 ceph-mon[31173]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:12:04.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:03 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2702469109' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:12:04.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:03 smithi092 ceph-mon[31173]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:12:04.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:03 smithi153 ceph-mon[35045]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:12:04.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:03 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2702469109' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:12:04.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:03 smithi153 ceph-mon[35045]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:12:04.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:03 smithi110 ceph-mon[34830]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:12:04.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:03 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2702469109' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:12:04.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:03 smithi110 ceph-mon[34830]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:12:04.387 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-11T22:12:04.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-11T22:12:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:04 smithi092 ceph-mon[31173]: pgmap v277: 97 pgs: 16 creating+peering, 38 unknown, 43 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:04 smithi092 ceph-mon[31173]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:12:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:04 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2702469109' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:12:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:04 smithi092 ceph-mon[31173]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:12:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:04 smithi092 ceph-mon[31173]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:12:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:04 smithi153 ceph-mon[35045]: pgmap v277: 97 pgs: 16 creating+peering, 38 unknown, 43 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:04 smithi153 ceph-mon[35045]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:12:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:04 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2702469109' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:12:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:04 smithi153 ceph-mon[35045]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:12:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:04 smithi153 ceph-mon[35045]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:12:05.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.145+0000 7f1b04c71640 1 -- 172.21.15.92:0/4106153717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b001082a0 msgr2=0x7f1b0010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:05.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.145+0000 7f1b04c71640 1 --2- 172.21.15.92:0/4106153717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b001082a0 0x7f1b0010ec40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009930 tx=0x7f1af802a420 comp rx=0 tx=0).stop 2024-09-11T22:12:05.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.145+0000 7f1b04c71640 1 -- 172.21.15.92:0/4106153717 shutdown_connections 2024-09-11T22:12:05.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.145+0000 7f1b04c71640 1 --2- 172.21.15.92:0/4106153717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b001082a0 0x7f1b0010ec40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.145+0000 7f1b04c71640 1 --2- 172.21.15.92:0/4106153717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b001036e0 0x7f1b00107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.145+0000 7f1b04c71640 1 --2- 172.21.15.92:0/4106153717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b00102d30 0x7f1b00103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.145+0000 7f1b04c71640 1 -- 172.21.15.92:0/4106153717 >> 172.21.15.92:0/4106153717 conn(0x7f1b000fc7d0 msgr2=0x7f1b000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 -- 172.21.15.92:0/4106153717 shutdown_connections 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 -- 172.21.15.92:0/4106153717 wait complete. 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 Processor -- start 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 -- start start 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b00102d30 0x7f1b00198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b001036e0 0x7f1b00198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b001082a0 0x7f1b0019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00111f50 con 0x7f1b00102d30 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00111dd0 con 0x7f1b001082a0 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.146+0000 7f1b04c71640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001120d0 con 0x7f1b001036e0 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b00102d30 0x7f1b00198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b00102d30 0x7f1b00198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47704/0 (socket says 172.21.15.92:47704) 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 -- 172.21.15.92:0/105875936 learned_addr learned my addr 172.21.15.92:0/105875936 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afdd9b640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b001036e0 0x7f1b00198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afed9d640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b001082a0 0x7f1b0019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 -- 172.21.15.92:0/105875936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b001036e0 msgr2=0x7f1b00198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b001036e0 0x7f1b00198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 -- 172.21.15.92:0/105875936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b001082a0 msgr2=0x7f1b0019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b001082a0 0x7f1b0019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 -- 172.21.15.92:0/105875936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0019d3d0 con 0x7f1b00102d30 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1afe59c640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b00102d30 0x7f1b00198420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1af000cce0 tx=0x7f1af000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1aef7fe640 1 -- 172.21.15.92:0/105875936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af000f650 con 0x7f1b00102d30 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0019d6c0 con 0x7f1b00102d30 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1aef7fe640 1 -- 172.21.15.92:0/105875936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1af000fca0 con 0x7f1b00102d30 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1aef7fe640 1 -- 172.21.15.92:0/105875936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0018b90 con 0x7f1b00102d30 2024-09-11T22:12:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.147+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001a4fa0 con 0x7f1b00102d30 2024-09-11T22:12:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.148+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b00103110 con 0x7f1b00102d30 2024-09-11T22:12:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.149+0000 7f1aef7fe640 1 -- 172.21.15.92:0/105875936 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1af000f7f0 con 0x7f1b00102d30 2024-09-11T22:12:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.150+0000 7f1aef7fe640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1ae80a2b80 0x7f1ae80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.150+0000 7f1afdd9b640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1ae80a2b80 0x7f1ae80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.150+0000 7f1aef7fe640 1 -- 172.21.15.92:0/105875936 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f1af00cf3e0 con 0x7f1b00102d30 2024-09-11T22:12:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.151+0000 7f1afdd9b640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1ae80a2b80 0x7f1ae80a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1b000fede0 tx=0x7f1af400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.154+0000 7f1aef7fe640 1 -- 172.21.15.92:0/105875936 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1af0014030 con 0x7f1b00102d30 2024-09-11T22:12:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:04 smithi110 ceph-mon[34830]: pgmap v277: 97 pgs: 16 creating+peering, 38 unknown, 43 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:04 smithi110 ceph-mon[34830]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:12:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:04 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2702469109' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:12:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:04 smithi110 ceph-mon[34830]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:12:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:04 smithi110 ceph-mon[34830]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:12:05.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.328+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f1b00199780 con 0x7f1b00102d30 2024-09-11T22:12:05.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.774+0000 7f1aef7fe640 1 -- 172.21.15.92:0/105875936 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f1af009c490 con 0x7f1b00102d30 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1ae80a2b80 msgr2=0x7f1ae80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1ae80a2b80 0x7f1ae80a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1b000fede0 tx=0x7f1af400b390 comp rx=0 tx=0).stop 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b00102d30 msgr2=0x7f1b00198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b00102d30 0x7f1b00198420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1af000cce0 tx=0x7f1af000a620 comp rx=0 tx=0).stop 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 shutdown_connections 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1ae80a2b80 0x7f1ae80a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b001082a0 0x7f1b0019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b001036e0 0x7f1b00198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 --2- 172.21.15.92:0/105875936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b00102d30 0x7f1b00198420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 >> 172.21.15.92:0/105875936 conn(0x7f1b000fc7d0 msgr2=0x7f1b00105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 shutdown_connections 2024-09-11T22:12:05.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:05.777+0000 7f1b04c71640 1 -- 172.21.15.92:0/105875936 wait complete. 2024-09-11T22:12:06.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:05 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] up:active 2024-09-11T22:12:06.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:05 smithi153 ceph-mon[35045]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:06.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:05 smithi153 ceph-mon[35045]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:06.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:05 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/105875936' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T22:12:06.141 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:05 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] up:active 2024-09-11T22:12:06.141 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:05 smithi092 ceph-mon[31173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:06.142 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:05 smithi092 ceph-mon[31173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:06.142 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:05 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/105875936' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T22:12:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:05 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] up:active 2024-09-11T22:12:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:05 smithi110 ceph-mon[34830]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:05 smithi110 ceph-mon[34830]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:05 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/105875936' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T22:12:06.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T22:12:06.695 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:06 smithi153 ceph-mon[35045]: pgmap v279: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 6 op/s 2024-09-11T22:12:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:06 smithi153 ceph-mon[35045]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:12:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:06 smithi153 ceph-mon[35045]: Cluster is now healthy 2024-09-11T22:12:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:06 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/105875936' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T22:12:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:06 smithi153 ceph-mon[35045]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T22:12:07.129 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:06 smithi092 ceph-mon[31173]: pgmap v279: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 6 op/s 2024-09-11T22:12:07.129 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:06 smithi092 ceph-mon[31173]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:12:07.129 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:06 smithi092 ceph-mon[31173]: Cluster is now healthy 2024-09-11T22:12:07.129 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:06 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/105875936' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T22:12:07.129 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:06 smithi092 ceph-mon[31173]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T22:12:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:06 smithi110 ceph-mon[34830]: pgmap v279: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 6 op/s 2024-09-11T22:12:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:06 smithi110 ceph-mon[34830]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:12:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:06 smithi110 ceph-mon[34830]: Cluster is now healthy 2024-09-11T22:12:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:06 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/105875936' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T22:12:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:06 smithi110 ceph-mon[34830]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T22:12:07.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 -- 172.21.15.92:0/2495342002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 msgr2=0x7f844c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:07.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 --2- 172.21.15.92:0/2495342002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c101200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f843c009900 tx=0x7f843c02a480 comp rx=0 tx=0).stop 2024-09-11T22:12:07.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 -- 172.21.15.92:0/2495342002 shutdown_connections 2024-09-11T22:12:07.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 --2- 172.21.15.92:0/2495342002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f844c102ef0 0x7f844c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:07.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 --2- 172.21.15.92:0/2495342002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844c102570 0x7f844c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:07.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 --2- 172.21.15.92:0/2495342002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c101200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 -- 172.21.15.92:0/2495342002 >> 172.21.15.92:0/2495342002 conn(0x7f844c0fc7d0 msgr2=0x7f844c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 -- 172.21.15.92:0/2495342002 shutdown_connections 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.215+0000 7f8451e97640 1 -- 172.21.15.92:0/2495342002 wait complete. 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 Processor -- start 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 -- start start 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844c102570 0x7f844c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f844c102ef0 0x7f844c1a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f844c110e00 con 0x7f844c100e20 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f844c110c80 con 0x7f844c102ef0 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f8451e97640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f844c110f80 con 0x7f844c102570 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:07.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47724/0 (socket says 172.21.15.92:47724) 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 -- 172.21.15.92:0/255060921 learned_addr learned my addr 172.21.15.92:0/255060921 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844affd640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844c102570 0x7f844c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844bfff640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f844c102ef0 0x7f844c1a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 -- 172.21.15.92:0/255060921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844c102570 msgr2=0x7f844c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844c102570 0x7f844c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 -- 172.21.15.92:0/255060921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f844c102ef0 msgr2=0x7f844c1a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f844c102ef0 0x7f844c1a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844b7fe640 1 -- 172.21.15.92:0/255060921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c1a4830 con 0x7f844c100e20 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844affd640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844c102570 0x7f844c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.216+0000 7f844bfff640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f844c102ef0 0x7f844c1a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.217+0000 7f844b7fe640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c199270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f843c006010 tx=0x7f843c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.217+0000 7f8448ff9640 1 -- 172.21.15.92:0/255060921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c03f030 con 0x7f844c100e20 2024-09-11T22:12:07.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.217+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c1a4a00 con 0x7f844c100e20 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.217+0000 7f8448ff9640 1 -- 172.21.15.92:0/255060921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f843c003cb0 con 0x7f844c100e20 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.217+0000 7f8448ff9640 1 -- 172.21.15.92:0/255060921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c035070 con 0x7f844c100e20 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.217+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c1a4eb0 con 0x7f844c100e20 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.218+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c101200 con 0x7f844c100e20 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.219+0000 7f8448ff9640 1 -- 172.21.15.92:0/255060921 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f843c042030 con 0x7f844c100e20 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.219+0000 7f8448ff9640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f84340a2b80 0x7f84340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.220+0000 7f844affd640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f84340a2b80 0x7f84340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.220+0000 7f8448ff9640 1 -- 172.21.15.92:0/255060921 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f843c0ef600 con 0x7f844c100e20 2024-09-11T22:12:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.220+0000 7f844affd640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f84340a2b80 0x7f84340a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f844c06aac0 tx=0x7f8440005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:07.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.224+0000 7f8448ff9640 1 -- 172.21.15.92:0/255060921 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f843c0bc530 con 0x7f844c100e20 2024-09-11T22:12:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:07.389+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f844c109090 con 0x7f844c100e20 2024-09-11T22:12:08.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:07 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/255060921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:12:08.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:07 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/255060921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:12:08.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:07 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/255060921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:12:08.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.792+0000 7f8448ff9640 1 -- 172.21.15.92:0/255060921 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f843c0bbe30 con 0x7f844c100e20 2024-09-11T22:12:08.793 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data_ec' created 2024-09-11T22:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.795+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f84340a2b80 msgr2=0x7f84340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.795+0000 7f8451e97640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f84340a2b80 0x7f84340a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f844c06aac0 tx=0x7f8440005d90 comp rx=0 tx=0).stop 2024-09-11T22:12:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.795+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 msgr2=0x7f844c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.795+0000 7f8451e97640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c199270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f843c006010 tx=0x7f843c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 shutdown_connections 2024-09-11T22:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f84340a2b80 0x7f84340a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f844c102ef0 0x7f844c1a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844c102570 0x7f844c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 --2- 172.21.15.92:0/255060921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c100e20 0x7f844c199270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 >> 172.21.15.92:0/255060921 conn(0x7f844c0fc7d0 msgr2=0x7f844c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:08.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 shutdown_connections 2024-09-11T22:12:08.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:08.796+0000 7f8451e97640 1 -- 172.21.15.92:0/255060921 wait complete. 2024-09-11T22:12:09.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:08 smithi153 ceph-mon[35045]: pgmap v281: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 862 B/s wr, 5 op/s 2024-09-11T22:12:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:08 smithi153 ceph-mon[35045]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T22:12:09.158 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:08 smithi092 ceph-mon[31173]: pgmap v281: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 862 B/s wr, 5 op/s 2024-09-11T22:12:09.158 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:08 smithi092 ceph-mon[31173]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T22:12:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:08 smithi110 ceph-mon[34830]: pgmap v281: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 862 B/s wr, 5 op/s 2024-09-11T22:12:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:08 smithi110 ceph-mon[34830]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T22:12:09.217 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:12:08 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: 2024-09-11T22:12:08.756+0000 7f818248d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T22:12:09.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-11T22:12:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:09.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:09 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/255060921' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:12:09.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:09 smithi110 ceph-mon[34830]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T22:12:10.138 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:09 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/255060921' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:12:10.138 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:09 smithi092 ceph-mon[31173]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 -- 172.21.15.92:0/2090358839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40100b30 msgr2=0x7f0d40100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 --2- 172.21.15.92:0/2090358839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40100b30 0x7f0d40100f10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009900 tx=0x7f0d3c02a470 comp rx=0 tx=0).stop 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 -- 172.21.15.92:0/2090358839 shutdown_connections 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 --2- 172.21.15.92:0/2090358839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d40105f40 0x7f0d401063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 --2- 172.21.15.92:0/2090358839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40101450 0x7f0d40105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 --2- 172.21.15.92:0/2090358839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40100b30 0x7f0d40100f10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 -- 172.21.15.92:0/2090358839 >> 172.21.15.92:0/2090358839 conn(0x7f0d400fc7a0 msgr2=0x7f0d400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 -- 172.21.15.92:0/2090358839 shutdown_connections 2024-09-11T22:12:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.221+0000 7f0d47594640 1 -- 172.21.15.92:0/2090358839 wait complete. 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 Processor -- start 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 -- start start 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d40100b30 0x7f0d401a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40101450 0x7f0d401a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40105f40 0x7f0d4019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401184f0 con 0x7f0d40105f40 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d40118370 con 0x7f0d40100b30 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d47594640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d40118670 con 0x7f0d40101450 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40105f40 0x7f0d4019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40105f40 0x7f0d4019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47738/0 (socket says 172.21.15.92:47738) 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 -- 172.21.15.92:0/1672921167 learned_addr learned my addr 172.21.15.92:0/1672921167 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45330640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d40100b30 0x7f0d401a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d44b2f640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40101450 0x7f0d401a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 -- 172.21.15.92:0/1672921167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40101450 msgr2=0x7f0d401a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40101450 0x7f0d401a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 -- 172.21.15.92:0/1672921167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d40100b30 msgr2=0x7f0d401a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d40100b30 0x7f0d401a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.222+0000 7f0d45b31640 1 -- 172.21.15.92:0/1672921167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4019f030 con 0x7f0d40105f40 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.223+0000 7f0d45b31640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40105f40 0x7f0d4019e900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800b970 tx=0x7f0d3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.223+0000 7f0d367fc640 1 -- 172.21.15.92:0/1672921167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38019070 con 0x7f0d40105f40 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.223+0000 7f0d367fc640 1 -- 172.21.15.92:0/1672921167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d38014440 con 0x7f0d40105f40 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.223+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4019f320 con 0x7f0d40105f40 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.223+0000 7f0d367fc640 1 -- 172.21.15.92:0/1672921167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d380075d0 con 0x7f0d40105f40 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.223+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401ab610 con 0x7f0d40105f40 2024-09-11T22:12:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.224+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d401059a0 con 0x7f0d40105f40 2024-09-11T22:12:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.226+0000 7f0d367fc640 1 -- 172.21.15.92:0/1672921167 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d3801c070 con 0x7f0d40105f40 2024-09-11T22:12:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.226+0000 7f0d367fc640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.226+0000 7f0d45330640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.227+0000 7f0d367fc640 1 -- 172.21.15.92:0/1672921167 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f0d380cece0 con 0x7f0d40105f40 2024-09-11T22:12:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.227+0000 7f0d45330640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c000c00 tx=0x7f0d3c005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.230+0000 7f0d367fc640 1 -- 172.21.15.92:0/1672921167 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d3809b3d0 con 0x7f0d40105f40 2024-09-11T22:12:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:09 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/255060921' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:12:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:09 smithi153 ceph-mon[35045]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T22:12:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.394+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f0d4019fe30 con 0x7f0d40105f40 2024-09-11T22:12:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.803+0000 7f0d367fc640 1 -- 172.21.15.92:0/1672921167 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f0d4019fe30 con 0x7f0d40105f40 2024-09-11T22:12:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 4 allow_ec_overwrites to true 2024-09-11T22:12:10.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d2c0a2b80 msgr2=0x7f0d2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:10.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c000c00 tx=0x7f0d3c005e80 comp rx=0 tx=0).stop 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40105f40 msgr2=0x7f0d4019e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40105f40 0x7f0d4019e900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800b970 tx=0x7f0d3800be30 comp rx=0 tx=0).stop 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 shutdown_connections 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d40105f40 0x7f0d4019e900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40101450 0x7f0d401a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 --2- 172.21.15.92:0/1672921167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d40100b30 0x7f0d401a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.806+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 >> 172.21.15.92:0/1672921167 conn(0x7f0d400fc7a0 msgr2=0x7f0d4010f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.807+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 shutdown_connections 2024-09-11T22:12:10.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:10.807+0000 7f0d47594640 1 -- 172.21.15.92:0/1672921167 wait complete. 2024-09-11T22:12:11.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:10 smithi092 ceph-mon[31173]: pgmap v284: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-11T22:12:11.126 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:10 smithi092 ceph-mon[31173]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:12:11.126 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:10 smithi092 ceph-mon[31173]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T22:12:11.126 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:10 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1672921167' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-11T22:12:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:10 smithi110 ceph-mon[34830]: pgmap v284: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-11T22:12:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:10 smithi110 ceph-mon[34830]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:12:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:10 smithi110 ceph-mon[34830]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T22:12:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:10 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1672921167' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-11T22:12:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:10 smithi153 ceph-mon[35045]: pgmap v284: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-11T22:12:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:10 smithi153 ceph-mon[35045]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:12:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:10 smithi153 ceph-mon[35045]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T22:12:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:10 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1672921167' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-11T22:12:11.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-11T22:12:11.727 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:12.155 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:11 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1672921167' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T22:12:12.155 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:11 smithi092 ceph-mon[31173]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T22:12:12.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:11 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1672921167' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T22:12:12.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:11 smithi110 ceph-mon[34830]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T22:12:12.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 -- 172.21.15.92:0/4236104475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 msgr2=0x7fec5810eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:12.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 --2- 172.21.15.92:0/4236104475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec5810eb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec50009cd0 tx=0x7fec5002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 -- 172.21.15.92:0/4236104475 shutdown_connections 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 --2- 172.21.15.92:0/4236104475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec5810eb00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 --2- 172.21.15.92:0/4236104475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58101290 0x7fec58107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 --2- 172.21.15.92:0/4236104475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec581008e0 0x7fec58100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 -- 172.21.15.92:0/4236104475 >> 172.21.15.92:0/4236104475 conn(0x7fec58077210 msgr2=0x7fec58077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 -- 172.21.15.92:0/4236104475 shutdown_connections 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 -- 172.21.15.92:0/4236104475 wait complete. 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 Processor -- start 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.235+0000 7fec607f3640 1 -- start start 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec581008e0 0x7fec58101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58101290 0x7fec58101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec581022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec58111e80 con 0x7fec58108160 2024-09-11T22:12:12.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec58111d00 con 0x7fec58101290 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec58112000 con 0x7fec581008e0 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec581022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec581022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47752/0 (socket says 172.21.15.92:47752) 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 -- 172.21.15.92:0/1276704632 learned_addr learned my addr 172.21.15.92:0/1276704632 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5e58f640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec581008e0 0x7fec58101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5dd8e640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58101290 0x7fec58101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 -- 172.21.15.92:0/1276704632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec581008e0 msgr2=0x7fec58101860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec581008e0 0x7fec58101860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 -- 172.21.15.92:0/1276704632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58101290 msgr2=0x7fec58101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58101290 0x7fec58101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 -- 172.21.15.92:0/1276704632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581a6860 con 0x7fec58108160 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5dd8e640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58101290 0x7fec58101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:12.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec5ed90640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec581022e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec50005cc0 tx=0x7fec50003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec4b7fe640 1 -- 172.21.15.92:0/1276704632 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50004450 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec4b7fe640 1 -- 172.21.15.92:0/1276704632 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec50031c90 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec581a6af0 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec4b7fe640 1 -- 172.21.15.92:0/1276704632 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5003b400 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.236+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec581a7030 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.237+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58100cc0 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.239+0000 7fec4b7fe640 1 -- 172.21.15.92:0/1276704632 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fec50004680 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.240+0000 7fec4b7fe640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec440a2b80 0x7fec440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.240+0000 7fec5e58f640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec440a2b80 0x7fec440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.240+0000 7fec4b7fe640 1 -- 172.21.15.92:0/1276704632 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fec5002e030 con 0x7fec58108160 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.240+0000 7fec5e58f640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec440a2b80 0x7fec440a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005fd0 tx=0x7fec4c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:12.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.243+0000 7fec4b7fe640 1 -- 172.21.15.92:0/1276704632 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec500bd790 con 0x7fec58108160 2024-09-11T22:12:12.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:11 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1672921167' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T22:12:12.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:11 smithi153 ceph-mon[35045]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T22:12:12.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.447+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fec5810a170 con 0x7fec58108160 2024-09-11T22:12:12.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.810+0000 7fec4b7fe640 1 -- 172.21.15.92:0/1276704632 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fec500bd090 con 0x7fec58108160 2024-09-11T22:12:12.811 INFO:teuthology.orchestra.run.smithi092.stderr:added data pool 4 to fsmap 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec440a2b80 msgr2=0x7fec440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec440a2b80 0x7fec440a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005fd0 tx=0x7fec4c005850 comp rx=0 tx=0).stop 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 msgr2=0x7fec581022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec581022e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec50005cc0 tx=0x7fec50003c20 comp rx=0 tx=0).stop 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 shutdown_connections 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec440a2b80 0x7fec440a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58108160 0x7fec581022e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58101290 0x7fec58101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 --2- 172.21.15.92:0/1276704632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec581008e0 0x7fec58101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:12.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 >> 172.21.15.92:0/1276704632 conn(0x7fec58077210 msgr2=0x7fec58105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:12.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 shutdown_connections 2024-09-11T22:12:12.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:12.813+0000 7fec607f3640 1 -- 172.21.15.92:0/1276704632 wait complete. 2024-09-11T22:12:13.109 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:12 smithi092 ceph-mon[31173]: pgmap v287: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-11T22:12:13.109 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:12 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1276704632' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-11T22:12:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:12 smithi110 ceph-mon[34830]: pgmap v287: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-11T22:12:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:12 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1276704632' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-11T22:12:13.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:12 smithi153 ceph-mon[35045]: pgmap v287: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-11T22:12:13.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:12 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1276704632' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-11T22:12:13.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:12:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:14.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:13 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1276704632' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T22:12:14.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:13 smithi092 ceph-mon[31173]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:12:14.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:13 smithi092 ceph-mon[31173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:13 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1276704632' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T22:12:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:13 smithi110 ceph-mon[34830]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:12:14.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:13 smithi110 ceph-mon[34830]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 -- 172.21.15.92:0/3088221200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f81082a0 msgr2=0x7f63f810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 --2- 172.21.15.92:0/3088221200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f81082a0 0x7f63f810ec40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009cd0 tx=0x7f63f002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 -- 172.21.15.92:0/3088221200 shutdown_connections 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 --2- 172.21.15.92:0/3088221200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f81082a0 0x7f63f810ec40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 --2- 172.21.15.92:0/3088221200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f81036e0 0x7f63f8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 --2- 172.21.15.92:0/3088221200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63f8102d30 0x7f63f8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 -- 172.21.15.92:0/3088221200 >> 172.21.15.92:0/3088221200 conn(0x7f63f80fc7d0 msgr2=0x7f63f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 -- 172.21.15.92:0/3088221200 shutdown_connections 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 -- 172.21.15.92:0/3088221200 wait complete. 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 Processor -- start 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.262+0000 7f63ff711640 1 -- start start 2024-09-11T22:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f8102d30 0x7f63f8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f81036e0 0x7f63f8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63f81082a0 0x7f63f8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8111f50 con 0x7f63f8102d30 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8111dd0 con 0x7f63f81082a0 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81120d0 con 0x7f63f81036e0 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f8102d30 0x7f63f8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f8102d30 0x7f63f8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47766/0 (socket says 172.21.15.92:47766) 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 -- 172.21.15.92:0/1100206269 learned_addr learned my addr 172.21.15.92:0/1100206269 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 -- 172.21.15.92:0/1100206269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f81036e0 msgr2=0x7f63f8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f81036e0 0x7f63f8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 -- 172.21.15.92:0/1100206269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63f81082a0 msgr2=0x7f63f8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63f81082a0 0x7f63f8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 -- 172.21.15.92:0/1100206269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f81a69b0 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63fd4ad640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f8102d30 0x7f63f8100990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c890 tx=0x7f63ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ea7fc640 1 -- 172.21.15.92:0/1100206269 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec004220 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ea7fc640 1 -- 172.21.15.92:0/1100206269 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63ec0043c0 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f81a6ca0 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ea7fc640 1 -- 172.21.15.92:0/1100206269 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec004a00 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.263+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f81a7190 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.264+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f8103110 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.265+0000 7f63ea7fc640 1 -- 172.21.15.92:0/1100206269 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63ec004ba0 con 0x7f63f8102d30 2024-09-11T22:12:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.266+0000 7f63ea7fc640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f63e40a2b80 0x7f63e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.266+0000 7f63fccac640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f63e40a2b80 0x7f63e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.266+0000 7f63ea7fc640 1 -- 172.21.15.92:0/1100206269 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f63ec0ceff0 con 0x7f63f8102d30 2024-09-11T22:12:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.267+0000 7f63fccac640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f63e40a2b80 0x7f63e40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63f8102420 tx=0x7f63f4005b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.270+0000 7f63ea7fc640 1 -- 172.21.15.92:0/1100206269 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63ec014030 con 0x7f63f8102d30 2024-09-11T22:12:14.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:13 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1276704632' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T22:12:14.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:13 smithi153 ceph-mon[35045]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:12:14.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:13 smithi153 ceph-mon[35045]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:14.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.470+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63f810a2e0 con 0x7f63f8102d30 2024-09-11T22:12:14.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.471+0000 7f63ea7fc640 1 -- 172.21.15.92:0/1100206269 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11845 (secure 0 0 0) 0x7f63ec09bd10 con 0x7f63f8102d30 2024-09-11T22:12:14.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:14.473 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"btime":"2024-09-11T22:12:12:809824+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:12.449462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24455,"qdb_cluster":[24455]},"id":1}]} 2024-09-11T22:12:14.473 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-09-11T22:12:14.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f63e40a2b80 msgr2=0x7f63e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:14.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f63e40a2b80 0x7f63e40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63f8102420 tx=0x7f63f4005b60 comp rx=0 tx=0).stop 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f8102d30 msgr2=0x7f63f8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f8102d30 0x7f63f8100990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c890 tx=0x7f63ec00cd50 comp rx=0 tx=0).stop 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 shutdown_connections 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f63e40a2b80 0x7f63e40a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63f81082a0 0x7f63f8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f81036e0 0x7f63f8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 --2- 172.21.15.92:0/1100206269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63f8102d30 0x7f63f8100990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.473+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 >> 172.21.15.92:0/1100206269 conn(0x7f63f80fc7d0 msgr2=0x7f63f8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.474+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 shutdown_connections 2024-09-11T22:12:14.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:14.474+0000 7f63ff711640 1 -- 172.21.15.92:0/1100206269 wait complete. 2024-09-11T22:12:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:14 smithi092 ceph-mon[31173]: pgmap v289: 129 pgs: 8 creating+peering, 3 unknown, 118 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-11T22:12:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:14 smithi092 ceph-mon[31173]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T22:12:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:14 smithi092 ceph-mon[31173]: Cluster is now healthy 2024-09-11T22:12:15.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:14 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1100206269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:15.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:12:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:14 smithi110 ceph-mon[34830]: pgmap v289: 129 pgs: 8 creating+peering, 3 unknown, 118 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-11T22:12:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:14 smithi110 ceph-mon[34830]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T22:12:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:14 smithi110 ceph-mon[34830]: Cluster is now healthy 2024-09-11T22:12:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:14 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1100206269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:14 smithi153 ceph-mon[35045]: pgmap v289: 129 pgs: 8 creating+peering, 3 unknown, 118 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-11T22:12:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:14 smithi153 ceph-mon[35045]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T22:12:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:14 smithi153 ceph-mon[35045]: Cluster is now healthy 2024-09-11T22:12:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:14 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1100206269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:15.350 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1210268094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90102570 msgr2=0x7f1f901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1210268094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90102570 0x7f1f901029b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009900 tx=0x7f1f8402a480 comp rx=0 tx=0).stop 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1210268094 shutdown_connections 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1210268094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90102ef0 0x7f1f9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1210268094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90102570 0x7f1f901029b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1210268094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f90100e20 0x7f1f90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1210268094 >> 172.21.15.92:0/1210268094 conn(0x7f1f900fc7d0 msgr2=0x7f1f900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1210268094 shutdown_connections 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1210268094 wait complete. 2024-09-11T22:12:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.848+0000 7f1f97e9b640 1 Processor -- start 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f97e9b640 1 -- start start 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f97e9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90100e20 0x7f1f900fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f97e9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f90102570 0x7f1f900fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f97e9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90102ef0 0x7f1f900fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f97e9b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90110e30 con 0x7f1f90100e20 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f97e9b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90110cb0 con 0x7f1f90102ef0 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f97e9b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90110fb0 con 0x7f1f90102570 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90100e20 0x7f1f900fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90100e20 0x7f1f900fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38928/0 (socket says 172.21.15.92:38928) 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 -- 172.21.15.92:0/1284255605 learned_addr learned my addr 172.21.15.92:0/1284255605 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95436640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f90102570 0x7f1f900fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 -- 172.21.15.92:0/1284255605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f90102570 msgr2=0x7f1f900fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f90102570 0x7f1f900fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f96438640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90102ef0 0x7f1f900fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 -- 172.21.15.92:0/1284255605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90102ef0 msgr2=0x7f1f900fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90102ef0 0x7f1f900fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 -- 172.21.15.92:0/1284255605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f901a68b0 con 0x7f1f90100e20 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95436640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f90102570 0x7f1f900fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f96438640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90102ef0 0x7f1f900fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f95c37640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90100e20 0x7f1f900fcc80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c890 tx=0x7f1f8000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f8effd640 1 -- 172.21.15.92:0/1284255605 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80004220 con 0x7f1f90100e20 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f8effd640 1 -- 172.21.15.92:0/1284255605 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f800043c0 con 0x7f1f90100e20 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.849+0000 7f1f8effd640 1 -- 172.21.15.92:0/1284255605 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80004a00 con 0x7f1f90100e20 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.850+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f901a6ba0 con 0x7f1f90100e20 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.850+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f901a7090 con 0x7f1f90100e20 2024-09-11T22:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.851+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f90101200 con 0x7f1f90100e20 2024-09-11T22:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.852+0000 7f1f8effd640 1 -- 172.21.15.92:0/1284255605 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1f80004ba0 con 0x7f1f90100e20 2024-09-11T22:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.852+0000 7f1f8effd640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1f780a2c00 0x7f1f780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.853+0000 7f1f95436640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1f780a2c00 0x7f1f780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.853+0000 7f1f8effd640 1 -- 172.21.15.92:0/1284255605 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1f800cef70 con 0x7f1f90100e20 2024-09-11T22:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.853+0000 7f1f95436640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1f780a2c00 0x7f1f780a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f900fe710 tx=0x7f1f84005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:15.857+0000 7f1f8effd640 1 -- 172.21.15.92:0/1284255605 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f80014030 con 0x7f1f90100e20 2024-09-11T22:12:16.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.028+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f900fe1b0 con 0x7f1f90100e20 2024-09-11T22:12:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.030+0000 7f1f8effd640 1 -- 172.21.15.92:0/1284255605 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7f1f8009bc90 con 0x7f1f90100e20 2024-09-11T22:12:16.031 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:16.031 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:12:16.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1f780a2c00 msgr2=0x7f1f780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:16.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1f780a2c00 0x7f1f780a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f900fe710 tx=0x7f1f84005a20 comp rx=0 tx=0).stop 2024-09-11T22:12:16.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90100e20 msgr2=0x7f1f900fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:16.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90100e20 0x7f1f900fcc80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c890 tx=0x7f1f8000cd50 comp rx=0 tx=0).stop 2024-09-11T22:12:16.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 shutdown_connections 2024-09-11T22:12:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1f780a2c00 0x7f1f780a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90102ef0 0x7f1f900fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f90102570 0x7f1f900fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 --2- 172.21.15.92:0/1284255605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f90100e20 0x7f1f900fcc80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 >> 172.21.15.92:0/1284255605 conn(0x7f1f900fc7d0 msgr2=0x7f1f90069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 shutdown_connections 2024-09-11T22:12:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:16.032+0000 7f1f97e9b640 1 -- 172.21.15.92:0/1284255605 wait complete. 2024-09-11T22:12:16.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:12:16.968 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:17.088 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:16 smithi092 ceph-mon[31173]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:17.088 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:16 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1284255605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:16 smithi110 ceph-mon[34830]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:16 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1284255605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:17.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:16 smithi153 ceph-mon[35045]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:17.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:16 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1284255605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:17.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/1311125258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102ef0 msgr2=0x7f7f1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:17.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/1311125258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102ef0 0x7f7f1810da40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009900 tx=0x7f7f0802a480 comp rx=0 tx=0).stop 2024-09-11T22:12:17.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/1311125258 shutdown_connections 2024-09-11T22:12:17.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/1311125258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102ef0 0x7f7f1810da40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/1311125258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f18102570 0x7f7f181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/1311125258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18100e20 0x7f7f18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/1311125258 >> 172.21.15.92:0/1311125258 conn(0x7f7f180fc7d0 msgr2=0x7f7f180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/1311125258 shutdown_connections 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.447+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/1311125258 wait complete. 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 Processor -- start 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 -- start start 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18100e20 0x7f7f181973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102570 0x7f7f18197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:17.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f18102ef0 0x7f7f1819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18110e70 con 0x7f7f18102570 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18110cf0 con 0x7f7f18102ef0 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f1cd6b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18110ff0 con 0x7f7f18100e20 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102570 0x7f7f18197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102570 0x7f7f18197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38952/0 (socket says 172.21.15.92:38952) 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.448+0000 7f7f15d9b640 1 -- 172.21.15.92:0/544453128 learned_addr learned my addr 172.21.15.92:0/544453128 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f1659c640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18100e20 0x7f7f181973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f16d9d640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f18102ef0 0x7f7f1819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f15d9b640 1 -- 172.21.15.92:0/544453128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18100e20 msgr2=0x7f7f181973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f15d9b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18100e20 0x7f7f181973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f15d9b640 1 -- 172.21.15.92:0/544453128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f18102ef0 msgr2=0x7f7f1819bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f15d9b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f18102ef0 0x7f7f1819bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f15d9b640 1 -- 172.21.15.92:0/544453128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1819c380 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f15d9b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102570 0x7f7f18197910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b4a0 tx=0x7f7f0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f0f7fe640 1 -- 172.21.15.92:0/544453128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00014070 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1819c670 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f0f7fe640 1 -- 172.21.15.92:0/544453128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f00010ac0 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f0f7fe640 1 -- 172.21.15.92:0/544453128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00017c30 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.449+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181a3f50 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.450+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f18101200 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.451+0000 7f7f0f7fe640 1 -- 172.21.15.92:0/544453128 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7f00017070 con 0x7f7f18102570 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.452+0000 7f7f0f7fe640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7f040a2b80 0x7f7f040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.452+0000 7f7f1659c640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7f040a2b80 0x7f7f040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.452+0000 7f7f0f7fe640 1 -- 172.21.15.92:0/544453128 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7f000ce890 con 0x7f7f18102570 2024-09-11T22:12:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.453+0000 7f7f1659c640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7f040a2b80 0x7f7f040a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7f10005fd0 tx=0x7f7f10009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.456+0000 7f7f0f7fe640 1 -- 172.21.15.92:0/544453128 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f0009af60 con 0x7f7f18102570 2024-09-11T22:12:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.621+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f18108b00 con 0x7f7f18102570 2024-09-11T22:12:17.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.622+0000 7f7f0f7fe640 1 -- 172.21.15.92:0/544453128 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7f7f0009f260 con 0x7f7f18102570 2024-09-11T22:12:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:17.624 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:12:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7f040a2b80 msgr2=0x7f7f040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7f040a2b80 0x7f7f040a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7f10005fd0 tx=0x7f7f10009680 comp rx=0 tx=0).stop 2024-09-11T22:12:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102570 msgr2=0x7f7f18197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102570 0x7f7f18197910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b4a0 tx=0x7f7f0000b960 comp rx=0 tx=0).stop 2024-09-11T22:12:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 shutdown_connections 2024-09-11T22:12:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7f040a2b80 0x7f7f040a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f18102ef0 0x7f7f1819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f18102570 0x7f7f18197910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 --2- 172.21.15.92:0/544453128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18100e20 0x7f7f181973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 >> 172.21.15.92:0/544453128 conn(0x7f7f180fc7d0 msgr2=0x7f7f180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 shutdown_connections 2024-09-11T22:12:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:17.625+0000 7f7f1cd6b640 1 -- 172.21.15.92:0/544453128 wait complete. 2024-09-11T22:12:17.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:17 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/544453128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:17 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/544453128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:18.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:12:18.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:17 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/544453128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:18.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:18 smithi092 ceph-mon[31173]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.911+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1633897245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102ef0 msgr2=0x7f45e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.911+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1633897245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102ef0 0x7f45e810da40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009ca0 tx=0x7f45e002a950 comp rx=0 tx=0).stop 2024-09-11T22:12:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.911+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1633897245 shutdown_connections 2024-09-11T22:12:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.911+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1633897245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102ef0 0x7f45e810da40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.911+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1633897245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8102570 0x7f45e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.911+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1633897245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e8100e20 0x7f45e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.911+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1633897245 >> 172.21.15.92:0/1633897245 conn(0x7f45e80fc7d0 msgr2=0x7f45e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.912+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1633897245 shutdown_connections 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.912+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1633897245 wait complete. 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.912+0000 7f45ef9f1640 1 Processor -- start 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.912+0000 7f45ef9f1640 1 -- start start 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ef9f1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8100e20 0x7f45e81973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ef9f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102570 0x7f45e81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ef9f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e8102ef0 0x7f45e819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ef9f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8110e50 con 0x7f45e8102570 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ef9f1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8110cd0 con 0x7f45e8100e20 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ef9f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8110fd0 con 0x7f45e8102ef0 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102570 0x7f45e81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102570 0x7f45e81978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38970/0 (socket says 172.21.15.92:38970) 2024-09-11T22:12:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 -- 172.21.15.92:0/1949318819 learned_addr learned my addr 172.21.15.92:0/1949318819 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45edf8e640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e8102ef0 0x7f45e819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ed78d640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8100e20 0x7f45e81973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 -- 172.21.15.92:0/1949318819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e8102ef0 msgr2=0x7f45e819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e8102ef0 0x7f45e819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 -- 172.21.15.92:0/1949318819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8100e20 msgr2=0x7f45e81973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8100e20 0x7f45e81973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 -- 172.21.15.92:0/1949318819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e819c360 con 0x7f45e8102570 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45ecf8c640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102570 0x7f45e81978f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c890 tx=0x7f45d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.913+0000 7f45de7fc640 1 -- 172.21.15.92:0/1949318819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8004220 con 0x7f45e8102570 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.914+0000 7f45de7fc640 1 -- 172.21.15.92:0/1949318819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45d80043c0 con 0x7f45e8102570 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.914+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e819c650 con 0x7f45e8102570 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.914+0000 7f45de7fc640 1 -- 172.21.15.92:0/1949318819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8004a00 con 0x7f45e8102570 2024-09-11T22:12:18.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.914+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e81a3f30 con 0x7f45e8102570 2024-09-11T22:12:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.915+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e8101200 con 0x7f45e8102570 2024-09-11T22:12:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.916+0000 7f45de7fc640 1 -- 172.21.15.92:0/1949318819 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f45d8004ba0 con 0x7f45e8102570 2024-09-11T22:12:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.916+0000 7f45de7fc640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f45d40a2b80 0x7f45d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.916+0000 7f45ed78d640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f45d40a2b80 0x7f45d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.916+0000 7f45de7fc640 1 -- 172.21.15.92:0/1949318819 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f45d80cef40 con 0x7f45e8102570 2024-09-11T22:12:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.917+0000 7f45ed78d640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f45d40a2b80 0x7f45d40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009890 tx=0x7f45e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:18.920+0000 7f45de7fc640 1 -- 172.21.15.92:0/1949318819 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45d8014030 con 0x7f45e8102570 2024-09-11T22:12:19.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.123+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45e8109090 con 0x7f45e8102570 2024-09-11T22:12:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.124+0000 7f45de7fc640 1 -- 172.21.15.92:0/1949318819 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11845 (secure 0 0 0) 0x7f45d809bc60 con 0x7f45e8102570 2024-09-11T22:12:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:19.125 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"btime":"2024-09-11T22:12:12:809824+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:12.449462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24455,"qdb_cluster":[24455]},"id":1}]} 2024-09-11T22:12:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-09-11T22:12:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f45d40a2b80 msgr2=0x7f45d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f45d40a2b80 0x7f45d40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009890 tx=0x7f45e4008040 comp rx=0 tx=0).stop 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102570 msgr2=0x7f45e81978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102570 0x7f45e81978f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c890 tx=0x7f45d800cd50 comp rx=0 tx=0).stop 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 shutdown_connections 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f45d40a2b80 0x7f45d40a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45e8102ef0 0x7f45e819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45e8102570 0x7f45e81978f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 --2- 172.21.15.92:0/1949318819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8100e20 0x7f45e81973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 >> 172.21.15.92:0/1949318819 conn(0x7f45e80fc7d0 msgr2=0x7f45e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 shutdown_connections 2024-09-11T22:12:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:19.126+0000 7f45ef9f1640 1 -- 172.21.15.92:0/1949318819 wait complete. 2024-09-11T22:12:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:18 smithi110 ceph-mon[34830]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:19.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:18 smithi153 ceph-mon[35045]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:19.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:12:20.002 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:19 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1949318819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:19 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1949318819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:20.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:19 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1949318819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:20.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 -- 172.21.15.92:0/316699678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 msgr2=0x7f3250101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:20.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 --2- 172.21.15.92:0/316699678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250101200 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3244009900 tx=0x7f324402a480 comp rx=0 tx=0).stop 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 -- 172.21.15.92:0/316699678 shutdown_connections 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 --2- 172.21.15.92:0/316699678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3250102ef0 0x7f325010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 --2- 172.21.15.92:0/316699678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250102570 0x7f32501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 --2- 172.21.15.92:0/316699678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250101200 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 -- 172.21.15.92:0/316699678 >> 172.21.15.92:0/316699678 conn(0x7f32500fc7d0 msgr2=0x7f32500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 -- 172.21.15.92:0/316699678 shutdown_connections 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.487+0000 7f3256fef640 1 -- 172.21.15.92:0/316699678 wait complete. 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 Processor -- start 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 -- start start 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3250102570 0x7f32501997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250102ef0 0x7f32501a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:20.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3250110e50 con 0x7f3250100e20 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3250110cd0 con 0x7f3250102570 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3256fef640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3250110fd0 con 0x7f3250102ef0 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38986/0 (socket says 172.21.15.92:38986) 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 -- 172.21.15.92:0/3778962893 learned_addr learned my addr 172.21.15.92:0/3778962893 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 -- 172.21.15.92:0/3778962893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250102ef0 msgr2=0x7f32501a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250102ef0 0x7f32501a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 -- 172.21.15.92:0/3778962893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3250102570 msgr2=0x7f32501997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3250102570 0x7f32501997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 -- 172.21.15.92:0/3778962893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32501a4820 con 0x7f3250100e20 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.488+0000 7f3254d8b640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250199260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f32440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.489+0000 7f324dffb640 1 -- 172.21.15.92:0/3778962893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324403f070 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.489+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32501a49f0 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.489+0000 7f324dffb640 1 -- 172.21.15.92:0/3778962893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3244003aa0 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.489+0000 7f324dffb640 1 -- 172.21.15.92:0/3778962893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244035030 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.489+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32501a4e40 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.490+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3250101200 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.491+0000 7f324dffb640 1 -- 172.21.15.92:0/3778962893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3244042070 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.491+0000 7f324dffb640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32400a2b80 0x7f32400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.492+0000 7f324ffff640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32400a2b80 0x7f32400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.492+0000 7f324dffb640 1 -- 172.21.15.92:0/3778962893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f32440efed0 con 0x7f3250100e20 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.492+0000 7f324ffff640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32400a2b80 0x7f32400a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f325006aac0 tx=0x7f323c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.495+0000 7f324dffb640 1 -- 172.21.15.92:0/3778962893 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32440bcbf0 con 0x7f3250100e20 2024-09-11T22:12:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.664+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f325006a560 con 0x7f3250100e20 2024-09-11T22:12:20.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.665+0000 7f324dffb640 1 -- 172.21.15.92:0/3778962893 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7f32440bc4f0 con 0x7f3250100e20 2024-09-11T22:12:20.667 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:20.667 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:12:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.667+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32400a2b80 msgr2=0x7f32400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.667+0000 7f3256fef640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32400a2b80 0x7f32400a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f325006aac0 tx=0x7f323c005c70 comp rx=0 tx=0).stop 2024-09-11T22:12:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.667+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 msgr2=0x7f3250199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.667+0000 7f3256fef640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250199260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f32440042f0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 shutdown_connections 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32400a2b80 0x7f32400a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3250102ef0 0x7f32501a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3250102570 0x7f32501997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 --2- 172.21.15.92:0/3778962893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3250100e20 0x7f3250199260 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 >> 172.21.15.92:0/3778962893 conn(0x7f32500fc7d0 msgr2=0x7f32500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 shutdown_connections 2024-09-11T22:12:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:20.668+0000 7f3256fef640 1 -- 172.21.15.92:0/3778962893 wait complete. 2024-09-11T22:12:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:20 smithi092 ceph-mon[31173]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:20 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3778962893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:20 smithi110 ceph-mon[34830]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:20 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3778962893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:21.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T22:12:03.420066+0000', 'modified': '2024-09-11T22:12:12.449462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24455': {'gid': 24455, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.110:6839/538708966', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 538708966}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 538708966}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24455, 'qdb_cluster': [24455]} 2024-09-11T22:12:21.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T22:12:21.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:12:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:20 smithi153 ceph-mon[35045]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:20 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3778962893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 -- 172.21.15.92:0/1693839815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102570 msgr2=0x7f710c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 --2- 172.21.15.92:0/1693839815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102570 0x7f710c1029b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009900 tx=0x7f70f402a480 comp rx=0 tx=0).stop 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 -- 172.21.15.92:0/1693839815 shutdown_connections 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 --2- 172.21.15.92:0/1693839815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c102ef0 0x7f710c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 --2- 172.21.15.92:0/1693839815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102570 0x7f710c1029b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 --2- 172.21.15.92:0/1693839815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f710c100e20 0x7f710c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 -- 172.21.15.92:0/1693839815 >> 172.21.15.92:0/1693839815 conn(0x7f710c0fc7d0 msgr2=0x7f710c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 -- 172.21.15.92:0/1693839815 shutdown_connections 2024-09-11T22:12:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.954+0000 7f7112df6640 1 -- 172.21.15.92:0/1693839815 wait complete. 2024-09-11T22:12:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.955+0000 7f7112df6640 1 Processor -- start 2024-09-11T22:12:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.955+0000 7f7112df6640 1 -- start start 2024-09-11T22:12:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.955+0000 7f7112df6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f710c100e20 0x7f710c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.955+0000 7f7112df6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c102570 0x7f710c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.955+0000 7f7112df6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102ef0 0x7f710c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7112df6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f710c110e30 con 0x7f710c102ef0 2024-09-11T22:12:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7112df6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f710c110cb0 con 0x7f710c102570 2024-09-11T22:12:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7112df6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f710c110fb0 con 0x7f710c100e20 2024-09-11T22:12:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102ef0 0x7f710c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102ef0 0x7f710c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39004/0 (socket says 172.21.15.92:39004) 2024-09-11T22:12:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 -- 172.21.15.92:0/1248495390 learned_addr learned my addr 172.21.15.92:0/1248495390 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7110b92640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f710c100e20 0x7f710c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f710bfff640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c102570 0x7f710c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 -- 172.21.15.92:0/1248495390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f710c100e20 msgr2=0x7f710c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f710c100e20 0x7f710c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 -- 172.21.15.92:0/1248495390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c102570 msgr2=0x7f710c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c102570 0x7f710c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 -- 172.21.15.92:0/1248495390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c19f570 con 0x7f710c102ef0 2024-09-11T22:12:21.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7110b92640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f710c100e20 0x7f710c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:21.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7111393640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102ef0 0x7f710c19ee90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f710400c970 tx=0x7f710400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:21.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7109ffb640 1 -- 172.21.15.92:0/1248495390 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104007bb0 con 0x7f710c102ef0 2024-09-11T22:12:21.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7109ffb640 1 -- 172.21.15.92:0/1248495390 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7104007d50 con 0x7f710c102ef0 2024-09-11T22:12:21.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c19f860 con 0x7f710c102ef0 2024-09-11T22:12:21.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.956+0000 7f7109ffb640 1 -- 172.21.15.92:0/1248495390 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104018700 con 0x7f710c102ef0 2024-09-11T22:12:21.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.957+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c1a7140 con 0x7f710c102ef0 2024-09-11T22:12:21.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.959+0000 7f7109ffb640 1 -- 172.21.15.92:0/1248495390 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f710400fa20 con 0x7f710c102ef0 2024-09-11T22:12:21.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.959+0000 7f71151cf740 1 Processor -- start 2024-09-11T22:12:21.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.959+0000 7f71151cf740 1 -- start start 2024-09-11T22:12:21.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.959+0000 7f7109ffb640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f80a2b80 0x7f70f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.959+0000 7f71151cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d00adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.959+0000 7f71151cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d015670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f710bfff640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f80a2b80 0x7f70f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f71151cf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d017fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7109ffb640 1 -- 172.21.15.92:0/1248495390 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f71040cf400 con 0x7f710c102ef0 2024-09-11T22:12:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f71151cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55656cf71090 con 0x55656d00a9d0 2024-09-11T22:12:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f71151cf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55656cf18a70 con 0x55656d015bb0 2024-09-11T22:12:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f71151cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55656cf6d050 con 0x55656d00b2f0 2024-09-11T22:12:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d00adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d00adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39008/0 (socket says 172.21.15.92:39008) 2024-09-11T22:12:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 -- 172.21.15.92:0/3481638970 learned_addr learned my addr 172.21.15.92:0/3481638970 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f710bfff640 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d015670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7111393640 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d017fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 -- 172.21.15.92:0/3481638970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 msgr2=0x55656d015670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d015670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 -- 172.21.15.92:0/3481638970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 msgr2=0x55656d017fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d017fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 -- 172.21.15.92:0/3481638970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656cf25130 con 0x55656d00a9d0 2024-09-11T22:12:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f710bfff640 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d015670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f710bfff640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f80a2b80 0x7f70f80a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f710c104c40 tx=0x7f70f4031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7110b92640 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d00adb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00af40 tx=0x7f70fc00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf44b7d713f38e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7102ffd640 1 -- 172.21.15.92:0/3481638970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc011850 con 0x55656d00a9d0 2024-09-11T22:12:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7102ffd640 1 -- 172.21.15.92:0/3481638970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70fc01a480 con 0x55656d00a9d0 2024-09-11T22:12:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.960+0000 7f7102ffd640 1 -- 172.21.15.92:0/3481638970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc010630 con 0x55656d00a9d0 2024-09-11T22:12:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 -- 172.21.15.92:0/3481638970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 msgr2=0x55656d00adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d00adb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00af40 tx=0x7f70fc00f7f0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 -- 172.21.15.92:0/3481638970 shutdown_connections 2024-09-11T22:12:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d017fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d015670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 --2- 172.21.15.92:0/3481638970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d00adb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 -- 172.21.15.92:0/3481638970 >> 172.21.15.92:0/3481638970 conn(0x55656d006f60 msgr2=0x55656d007340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 -- 172.21.15.92:0/3481638970 shutdown_connections 2024-09-11T22:12:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 -- 172.21.15.92:0/3481638970 wait complete. 2024-09-11T22:12:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.961+0000 7f71151cf740 1 -- start start 2024-09-11T22:12:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 1 Processor -- start 2024-09-11T22:12:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d015410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d012760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d012ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55656d018780 con 0x55656d00a9d0 2024-09-11T22:12:21.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55656cf2bf20 con 0x55656d015bb0 2024-09-11T22:12:21.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55656d018900 con 0x55656d00b2f0 2024-09-11T22:12:21.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f71151cf740 10 client.0 _ll_register_callbacks cb 0x55656cef5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:12:21.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d015410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d015410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39016/0 (socket says 172.21.15.92:39016) 2024-09-11T22:12:21.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 -- 172.21.15.92:0/2046273415 learned_addr learned my addr 172.21.15.92:0/2046273415 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f710bfff640 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d012760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7111393640 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d012ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 -- 172.21.15.92:0/2046273415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 msgr2=0x55656d012760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d012760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 -- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 msgr2=0x55656d012ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d012ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 -- 172.21.15.92:0/2046273415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656cce22b0 con 0x55656d00a9d0 2024-09-11T22:12:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f7110b92640 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d015410 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ac80 tx=0x7f70fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2634ee3a221a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.962+0000 7f70ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:12:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f70ebfff640 1 -- 172.21.15.92:0/2046273415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc01a840 con 0x55656d00a9d0 2024-09-11T22:12:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc01a480 con 0x55656d00a9d0 2024-09-11T22:12:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f70ebfff640 1 -- 172.21.15.92:0/2046273415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f70fc004b20 con 0x55656d00a9d0 2024-09-11T22:12:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f70ebfff640 1 -- 172.21.15.92:0/2046273415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f70fc004d60 con 0x55656d00a9d0 2024-09-11T22:12:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 10 client.14727 fetch_fsmap learned FSMap version 17 2024-09-11T22:12:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55656d0240c0 con 0x55656d00a9d0 2024-09-11T22:12:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f70ebfff640 1 -- 172.21.15.92:0/2046273415 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70fc01a480 con 0x55656d00a9d0 2024-09-11T22:12:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f70ebfff640 1 -- 172.21.15.92:0/2046273415 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f70fc0050b0 con 0x55656d00a9d0 2024-09-11T22:12:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 10 client.14727 fetch_fsmap finished waiting for FSMap version 17 2024-09-11T22:12:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 10 client.14727 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:12:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 20 client.14727 populate_metadata read hostname 'smithi092' 2024-09-11T22:12:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d026350 0x55656d0277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 4 client.14727 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:12:21.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 --> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55656cf7e600 con 0x55656d026350 2024-09-11T22:12:21.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.963+0000 7f710bfff640 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d026350 0x55656d0277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:21.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.964+0000 7f70ebfff640 10 client.14727 ms_handle_connect on v2:172.21.15.110:6838/538708966 2024-09-11T22:12:21.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.964+0000 7f710bfff640 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d026350 0x55656d0277c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b76d2e17e776a5 server_cookie=290577e5a7efc637 in_seq=0 out_seq=0 2024-09-11T22:12:21.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f70ebfff640 1 -- 172.21.15.92:0/2046273415 <== mds.0 v2:172.21.15.110:6838/538708966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55656cf7e600 con 0x55656d026350 2024-09-11T22:12:21.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f70ebfff640 10 client.14727 handle_command_reply: tid=0 2024-09-11T22:12:21.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 client.14727 shutdown 2024-09-11T22:12:21.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 msgr2=0x55656d015410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:21.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d015410 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ac80 tx=0x7f70fc004ab0 comp rx=0 tx=0).stop 2024-09-11T22:12:21.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 shutdown_connections 2024-09-11T22:12:21.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d026350 0x55656d0277c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d015bb0 0x55656d012ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d012760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 --2- 172.21.15.92:0/2046273415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d00a9d0 0x55656d015410 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 >> 172.21.15.92:0/2046273415 conn(0x55656d006f60 msgr2=0x55656d00cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f70ebfff640 0 client.14727 ms_handle_reset on v2:172.21.15.110:6838/538708966 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 shutdown_connections 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 1 -- 172.21.15.92:0/2046273415 wait complete. 2024-09-11T22:12:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.965+0000 7f71151cf740 20 client.14727 trim_cache size 0 max 16384 2024-09-11T22:12:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 Processor -- start 2024-09-11T22:12:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 -- start start 2024-09-11T22:12:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d026350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55656d025b00 con 0x55656d015bb0 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55656d0261d0 con 0x55656d00a9d0 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f71151cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55656d0268a0 con 0x55656d00b2f0 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39028/0 (socket says 172.21.15.92:39028) 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 -- 172.21.15.92:0/1612692971 learned_addr learned my addr 172.21.15.92:0/1612692971 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 -- 172.21.15.92:0/1612692971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 msgr2=0x55656d00cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7110b92640 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d026350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f710bfff640 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 -- 172.21.15.92:0/1612692971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 msgr2=0x55656d026350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d026350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 -- 172.21.15.92:0/1612692971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656d015780 con 0x55656d015bb0 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7110b92640 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d026350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f710bfff640 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.983+0000 7f7111393640 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02faa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f710409b9b0 tx=0x7f71040975a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b67ff6a03fede7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71027fc640 1 -- 172.21.15.92:0/1612692971 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040b4020 con 0x55656d015bb0 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71027fc640 1 -- 172.21.15.92:0/1612692971 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71040ce040 con 0x55656d015bb0 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71027fc640 1 -- 172.21.15.92:0/1612692971 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040a4200 con 0x55656d015bb0 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 -- 172.21.15.92:0/1612692971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 msgr2=0x55656d02faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02faa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f710409b9b0 tx=0x7f71040975a0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 -- 172.21.15.92:0/1612692971 shutdown_connections 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02faa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 --2- 172.21.15.92:0/1612692971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d026350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 -- 172.21.15.92:0/1612692971 >> 172.21.15.92:0/1612692971 conn(0x55656d006f60 msgr2=0x55656d00c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 -- 172.21.15.92:0/1612692971 shutdown_connections 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 -- 172.21.15.92:0/1612692971 wait complete. 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.984+0000 7f71151cf740 1 -- start start 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 1 Processor -- start 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d00ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55656d027640 con 0x55656d015bb0 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55656d026f70 con 0x55656d00a9d0 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55656d027d10 con 0x55656d00b2f0 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f71151cf740 10 client.0 _ll_register_callbacks cb 0x7f710400a4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:22.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39034/0 (socket says 172.21.15.92:39034) 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 -- 172.21.15.92:0/902992058 learned_addr learned my addr 172.21.15.92:0/902992058 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f710bfff640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7110b92640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d00ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 -- 172.21.15.92:0/902992058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 msgr2=0x55656d00f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 -- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 msgr2=0x55656d00ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d00ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7111393640 1 -- 172.21.15.92:0/902992058 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656d00fba0 con 0x55656d015bb0 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.985+0000 7f7110b92640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d00ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f7111393640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02ea80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f710401f000 tx=0x7f710409fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0d9babdc9b76f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f70ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f70ebfff640 1 -- 172.21.15.92:0/902992058 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104064020 con 0x55656d015bb0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71040ce040 con 0x55656d015bb0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f70ebfff640 1 -- 172.21.15.92:0/902992058 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f71040a4400 con 0x55656d015bb0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f70ebfff640 1 -- 172.21.15.92:0/902992058 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f71040b4050 con 0x55656d015bb0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 10 client.14733 fetch_fsmap learned FSMap version 17 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55656d030030 con 0x55656d015bb0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f70ebfff640 1 -- 172.21.15.92:0/902992058 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71040ce040 con 0x55656d015bb0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f70ebfff640 1 -- 172.21.15.92:0/902992058 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f710409a8e0 con 0x55656d015bb0 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 10 client.14733 fetch_fsmap finished waiting for FSMap version 17 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 10 client.14733 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 20 client.14733 populate_metadata read hostname 'smithi092' 2024-09-11T22:12:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d094890 0x55656d0935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 4 client.14733 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.986+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 --> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] -- command(tid 0: {"prefix": "status"}) -- 0x55656cf78b10 con 0x55656d094890 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f7110b92640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d094890 0x55656d0935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f70ebfff640 10 client.14733 ms_handle_connect on v2:172.21.15.110:6838/538708966 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f7110b92640 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d094890 0x55656d0935a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda9a283a13555e5 server_cookie=36f5f78440c3157a in_seq=0 out_seq=0 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f70ebfff640 1 -- 172.21.15.92:0/902992058 <== mds.0 v2:172.21.15.110:6838/538708966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55656cf78b10 con 0x55656d094890 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f70ebfff640 10 client.14733 handle_command_reply: tid=0 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 client.14733 shutdown 2024-09-11T22:12:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 msgr2=0x55656d02ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02ea80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f710401f000 tx=0x7f710409fdd0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 shutdown_connections 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x55656d094890 0x55656d0935a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55656d015bb0 0x55656d02ea80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55656d00b2f0 0x55656d00f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f70ebfff640 0 client.14733 ms_handle_reset on v2:172.21.15.110:6838/538708966 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.987+0000 7f71151cf740 1 --2- 172.21.15.92:0/902992058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55656d00a9d0 0x55656d00ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 >> 172.21.15.92:0/902992058 conn(0x55656d006f60 msgr2=0x55656d00cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 shutdown_connections 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f71151cf740 1 -- 172.21.15.92:0/902992058 wait complete. 2024-09-11T22:12:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f71151cf740 20 client.14733 trim_cache size 0 max 16384 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24455, 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:12:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 18.222915360999998, 2024-09-11T22:12:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-09-11T22:12:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 90, 2024-09-11T22:12:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 89, 2024-09-11T22:12:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 29.034435286000001 2024-09-11T22:12:22.017 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:12:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f80a2b80 msgr2=0x7f70f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f80a2b80 0x7f70f80a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f710c104c40 tx=0x7f70f4031430 comp rx=0 tx=0).stop 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102ef0 msgr2=0x7f710c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102ef0 0x7f710c19ee90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f710400c970 tx=0x7f710400ce30 comp rx=0 tx=0).stop 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 shutdown_connections 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f80a2b80 0x7f70f80a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710c102ef0 0x7f710c19ee90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c102570 0x7f710c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.988+0000 7f7112df6640 1 --2- 172.21.15.92:0/1248495390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f710c100e20 0x7f710c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.989+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 >> 172.21.15.92:0/1248495390 conn(0x7f710c0fc7d0 msgr2=0x7f710c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.989+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 shutdown_connections 2024-09-11T22:12:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:21.989+0000 7f7112df6640 1 -- 172.21.15.92:0/1248495390 wait complete. 2024-09-11T22:12:22.701 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T22:12:22.701 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:22.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T22:12:22.946 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:23.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:22 smithi092 ceph-mon[31173]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:22 smithi110 ceph-mon[34830]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:22 smithi153 ceph-mon[35045]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:23.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 -- 172.21.15.92:0/273053256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4106f10 msgr2=0x7f99c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 --2- 172.21.15.92:0/273053256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4106f10 0x7f99c410ea90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009ca0 tx=0x7f99bc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 -- 172.21.15.92:0/273053256 shutdown_connections 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 --2- 172.21.15.92:0/273053256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4106f10 0x7f99c410ea90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 --2- 172.21.15.92:0/273053256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c4102790 0x7f99c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 --2- 172.21.15.92:0/273053256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99c4101de0 0x7f99c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 -- 172.21.15.92:0/273053256 >> 172.21.15.92:0/273053256 conn(0x7f99c40fd8c0 msgr2=0x7f99c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 -- 172.21.15.92:0/273053256 shutdown_connections 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.445+0000 7f99caa24640 1 -- 172.21.15.92:0/273053256 wait complete. 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 Processor -- start 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 -- start start 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c4101de0 0x7f99c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4102790 0x7f99c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:23.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99c4106f10 0x7f99c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4111ec0 con 0x7f99c4102790 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4111d40 con 0x7f99c4106f10 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99caa24640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4112040 con 0x7f99c4101de0 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4102790 0x7f99c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4102790 0x7f99c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39044/0 (socket says 172.21.15.92:39044) 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 -- 172.21.15.92:0/1451344599 learned_addr learned my addr 172.21.15.92:0/1451344599 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c3fff640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c4101de0 0x7f99c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c8fc1640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99c4106f10 0x7f99c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 -- 172.21.15.92:0/1451344599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c4101de0 msgr2=0x7f99c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c4101de0 0x7f99c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 -- 172.21.15.92:0/1451344599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99c4106f10 msgr2=0x7f99c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99c4106f10 0x7f99c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c37fe640 1 -- 172.21.15.92:0/1451344599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c41a6940 con 0x7f99c4102790 2024-09-11T22:12:23.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.446+0000 7f99c8fc1640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99c4106f10 0x7f99c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.447+0000 7f99c37fe640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4102790 0x7f99c4074a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b4a0 tx=0x7f99b800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.447+0000 7f99c17fa640 1 -- 172.21.15.92:0/1451344599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8015020 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.447+0000 7f99c17fa640 1 -- 172.21.15.92:0/1451344599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b8011ce0 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.447+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c41a6c30 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.447+0000 7f99c17fa640 1 -- 172.21.15.92:0/1451344599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8010dc0 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.447+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c41a7170 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.448+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c41021c0 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.449+0000 7f99c17fa640 1 -- 172.21.15.92:0/1451344599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f99b8018020 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.450+0000 7f99c17fa640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99ac0a2c00 0x7f99ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.450+0000 7f99c3fff640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99ac0a2c00 0x7f99ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.450+0000 7f99c17fa640 1 -- 172.21.15.92:0/1451344599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f99b80cf060 con 0x7f99c4102790 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.450+0000 7f99c3fff640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99ac0a2c00 0x7f99ac0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009990 tx=0x7f99b4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:23.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.453+0000 7f99c17fa640 1 -- 172.21.15.92:0/1451344599 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99b801f020 con 0x7f99c4102790 2024-09-11T22:12:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.655+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f99c410a0f0 con 0x7f99c4102790 2024-09-11T22:12:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.655+0000 7f99c17fa640 1 -- 172.21.15.92:0/1451344599 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f99b809bd80 con 0x7f99c4102790 2024-09-11T22:12:23.656 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-11T22:12:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99ac0a2c00 msgr2=0x7f99ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99ac0a2c00 0x7f99ac0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009990 tx=0x7f99b4009110 comp rx=0 tx=0).stop 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4102790 msgr2=0x7f99c4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4102790 0x7f99c4074a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b4a0 tx=0x7f99b800b8a0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 shutdown_connections 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99ac0a2c00 0x7f99ac0a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99c4106f10 0x7f99c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99c4102790 0x7f99c4074a30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 --2- 172.21.15.92:0/1451344599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c4101de0 0x7f99c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 >> 172.21.15.92:0/1451344599 conn(0x7f99c40fd8c0 msgr2=0x7f99c40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 shutdown_connections 2024-09-11T22:12:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:23.658+0000 7f99caa24640 1 -- 172.21.15.92:0/1451344599 wait complete. 2024-09-11T22:12:23.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:23 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1451344599' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:12:24.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:23 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1451344599' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:12:24.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:23 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1451344599' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:12:24.449 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T22:12:24.449 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T22:12:24.449 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-09-11T22:12:24.449 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-11T22:12:24.450 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:24.450 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T22:12:24.450 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T22:12:24.450 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T22:12:24.450 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-11T22:12:24.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:24.450 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-09-11T22:12:24.467 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T22:12:24.467 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T22:12:24.467 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T22:12:24.468 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:12:24.469 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T22:12:24.469 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42135sec preferred_lft 42135sec 2024-09-11T22:12:24.469 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-11T22:12:24.469 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:12:24.469 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T22:12:24.470 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:12:24.470 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:12:24.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2024-09-11T22:12:24.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2024-09-11T22:12:24.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2024-09-11T22:12:24.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T22:12:24.470 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:12:24.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:12:24.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:12:24.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:24.680 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T22:12:24.705 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-11T22:12:24.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:24.706 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-09-11T22:12:24.775 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-09-11T22:12:24.775 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T22:12:24.776 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T22:12:24.776 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T22:12:24.776 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T22:12:24.776 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:12:24.776 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:12:24.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T22:12:24.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T22:12:24.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T22:12:24.777 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:12:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:12:24.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:24 smithi092 ceph-mon[31173]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:25.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:12:25.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:25.091 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-11T22:12:25.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:25.106 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-09-11T22:12:25.161 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:12:25.161 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:12:25.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T22:12:25.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-11T22:12:25.161 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:12:25.190 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:24 smithi153 ceph-mon[35045]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:25.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:24 smithi110 ceph-mon[34830]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:25.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:12:25.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:12:25.289 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-11T22:12:25.290 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:12:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:12:25.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:12:25.450 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:12:25.450 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:12:25.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2024-09-11T22:12:25.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T22:12:25.450 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:12:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:12:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:12:25.529 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-11T22:12:25.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:12:25.529 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:25.544 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-11T22:12:25.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:12:25.545 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:25.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-11T22:12:25.665 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T22:12:25.722 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-11T22:12:25.722 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-11T22:12:25.722 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-11T22:12:25.722 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-11T22:12:25.722 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-11T22:12:25.722 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-11T22:12:25.722 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T22:12:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/205a3cf67e962caef53c7d9f16ee57f82186beeb3f77672d371a8524df0412e9/merged 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b43dca6d6c930599632e7cbc1ffc10421bb5773b5661239002606220f430c440/merged 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/0 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/149012ee1cfd020bbc35631c38da1b6c0dbef855b875ffafab34afc4f166aff0/merged 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b9cd808df467a5a77d7a9b9f24fbe0421769b37ff3f487c6e39991de98e0c4c3/merged 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e7d143339e84352e4cd39384d3592ea9059095e676b9f9939cf1da58ddfaf97a/merged 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/72d2b71c6241585d2292458bd21d5f0d5a1804796666a88d95e8ef6740532c7e/merged 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/439e56349dfaee14adfe3785a2e894297ce589a0e7a7ee20027fb9f0f054fee8/merged 2024-09-11T22:12:25.724 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/a53191da74c76582a71960335880c1792bc4f53c63be01d45dc8789f4502d30e/merged 2024-09-11T22:12:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e6b3cc7ab02e80c813880c96e15b6d432350447baa3c53da85fdc1e9928aed0e/merged 2024-09-11T22:12:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/dc98e095f781c132f81a0d6e06412b66681707091221fda195e4a3f25112c917/merged 2024-09-11T22:12:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-11T22:12:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T22:12:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T22:12:25.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:25.725 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-11T22:12:25.777 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-11T22:12:25.778 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-11T22:12:25.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-11T22:12:25.845 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T22:12:25.901 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-11T22:12:25.901 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-11T22:12:25.902 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/205a3cf67e962caef53c7d9f16ee57f82186beeb3f77672d371a8524df0412e9/merged 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b43dca6d6c930599632e7cbc1ffc10421bb5773b5661239002606220f430c440/merged 2024-09-11T22:12:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/0 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/149012ee1cfd020bbc35631c38da1b6c0dbef855b875ffafab34afc4f166aff0/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b9cd808df467a5a77d7a9b9f24fbe0421769b37ff3f487c6e39991de98e0c4c3/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e7d143339e84352e4cd39384d3592ea9059095e676b9f9939cf1da58ddfaf97a/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/72d2b71c6241585d2292458bd21d5f0d5a1804796666a88d95e8ef6740532c7e/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/439e56349dfaee14adfe3785a2e894297ce589a0e7a7ee20027fb9f0f054fee8/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/a53191da74c76582a71960335880c1792bc4f53c63be01d45dc8789f4502d30e/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e6b3cc7ab02e80c813880c96e15b6d432350447baa3c53da85fdc1e9928aed0e/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/dc98e095f781c132f81a0d6e06412b66681707091221fda195e4a3f25112c917/merged 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T22:12:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T22:12:25.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:25.905 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-11T22:12:25.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:2024-09-11T22:12:25.929+0000 7fe18ab35480 -1 init, newargv = 0x55fb78bdcb10 newargc=15 2024-09-11T22:12:25.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[76140]: starting ceph client 2024-09-11T22:12:25.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[76140]: starting fuse 2024-09-11T22:12:25.946 INFO:teuthology.orchestra.run.smithi092.stdout:120 2024-09-11T22:12:25.946 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-11T22:12:25.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2024-09-11T22:12:25.946 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2024-09-11T22:12:25.946 DEBUG:teuthology.orchestra.run.smithi092:> import re 2024-09-11T22:12:25.946 DEBUG:teuthology.orchestra.run.smithi092:> import os 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:12:25.947 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> return f 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.admin")) 2024-09-11T22:12:25.948 DEBUG:teuthology.orchestra.run.smithi092:> ' 2024-09-11T22:12:26.064 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.admin.76140.asok 2024-09-11T22:12:26.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:12:26.075 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76140.asok 2024-09-11T22:12:26.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:26.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76140.asok status 2024-09-11T22:12:26.210 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:12:26.210 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2024-09-11T22:12:26.210 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-11T22:12:26.210 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "admin", 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "76140", 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 14745, 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 14745 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:12:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1775115294 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1775115294 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.14745 192.168.0.1:0/1775115294", 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/1775115294", 2024-09-11T22:12:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2024-09-11T22:12:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 17, 2024-09-11T22:12:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 90, 2024-09-11T22:12:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2024-09-11T22:12:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2024-09-11T22:12:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2024-09-11T22:12:26.213 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:12:26.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:26.219 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2024-09-11T22:12:26.219 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:26.234 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2024-09-11T22:12:26.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:26.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:26.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:26.300 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-11T22:12:26.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T22:12:26.409 INFO:teuthology.orchestra.run.smithi092.stdout:# file: . 2024-09-11T22:12:26.409 INFO:teuthology.orchestra.run.smithi092.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-11T22:12:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:12:26.415 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:12:26.415 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:12:26.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T22:12:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:27.181 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:26 smithi092 ceph-mon[31173]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:26 smithi110 ceph-mon[34830]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:26 smithi153 ceph-mon[35045]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:27.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.261+0000 7fd1cf288640 1 -- 172.21.15.92:0/2762428075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81082a0 msgr2=0x7fd1c810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:27.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.261+0000 7fd1cf288640 1 --2- 172.21.15.92:0/2762428075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81082a0 0x7fd1c810ec40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009cd0 tx=0x7fd1c402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 -- 172.21.15.92:0/2762428075 shutdown_connections 2024-09-11T22:12:27.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 --2- 172.21.15.92:0/2762428075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81082a0 0x7fd1c810ec40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 --2- 172.21.15.92:0/2762428075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c81036e0 0x7fd1c8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 --2- 172.21.15.92:0/2762428075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1c8102d30 0x7fd1c8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 -- 172.21.15.92:0/2762428075 >> 172.21.15.92:0/2762428075 conn(0x7fd1c80fc7d0 msgr2=0x7fd1c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 -- 172.21.15.92:0/2762428075 shutdown_connections 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 -- 172.21.15.92:0/2762428075 wait complete. 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 Processor -- start 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.262+0000 7fd1cf288640 1 -- start start 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1c8102d30 0x7fd1c8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81036e0 0x7fd1c8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c81082a0 0x7fd1c819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8111f50 con 0x7fd1c81036e0 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8111dd0 con 0x7fd1c8102d30 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c81120d0 con 0x7fd1c81082a0 2024-09-11T22:12:27.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81036e0 0x7fd1c8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81036e0 0x7fd1c8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51216/0 (socket says 172.21.15.92:51216) 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 -- 172.21.15.92:0/824986867 learned_addr learned my addr 172.21.15.92:0/824986867 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cd825640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c81082a0 0x7fd1c819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cd024640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1c8102d30 0x7fd1c8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 -- 172.21.15.92:0/824986867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c81082a0 msgr2=0x7fd1c819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c81082a0 0x7fd1c819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 -- 172.21.15.92:0/824986867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1c8102d30 msgr2=0x7fd1c8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1c8102d30 0x7fd1c8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 -- 172.21.15.92:0/824986867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c819d3d0 con 0x7fd1c81036e0 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cd024640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1c8102d30 0x7fd1c8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cc823640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81036e0 0x7fd1c8198960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800e9c0 tx=0x7fd1b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1be7fc640 1 -- 172.21.15.92:0/824986867 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800ccc0 con 0x7fd1c81036e0 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c819d6c0 con 0x7fd1c81036e0 2024-09-11T22:12:27.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1be7fc640 1 -- 172.21.15.92:0/824986867 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1b80054d0 con 0x7fd1c81036e0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1be7fc640 1 -- 172.21.15.92:0/824986867 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8010430 con 0x7fd1c81036e0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.263+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c81a4fa0 con 0x7fd1c81036e0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.264+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c8103110 con 0x7fd1c81036e0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.266+0000 7fd1be7fc640 1 -- 172.21.15.92:0/824986867 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd1b8005000 con 0x7fd1c81036e0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.267+0000 7fd1be7fc640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd1b40a2b80 0x7fd1b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.267+0000 7fd1be7fc640 1 -- 172.21.15.92:0/824986867 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd1b80cfb70 con 0x7fd1c81036e0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.267+0000 7fd1cd024640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd1b40a2b80 0x7fd1b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.268+0000 7fd1cd024640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd1b40a2b80 0x7fd1b40a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000baa0 tx=0x7fd1c0005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.270+0000 7fd1be7fc640 1 -- 172.21.15.92:0/824986867 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1b80052f0 con 0x7fd1c81036e0 2024-09-11T22:12:27.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.448+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1c810a2b0 con 0x7fd1c81036e0 2024-09-11T22:12:27.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.448+0000 7fd1be7fc640 1 -- 172.21.15.92:0/824986867 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+589 (secure 0 0 0) 0x7fd1b809c890 con 0x7fd1c81036e0 2024-09-11T22:12:27.449 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2458669467 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2872938890 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3563316301 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3468435378 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/125981315 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/4186575934 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:27.450 INFO:teuthology.orchestra.run.smithi092.stderr:listed 10 entries 2024-09-11T22:12:27.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd1b40a2b80 msgr2=0x7fd1b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:27.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd1b40a2b80 0x7fd1b40a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000baa0 tx=0x7fd1c0005d90 comp rx=0 tx=0).stop 2024-09-11T22:12:27.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81036e0 msgr2=0x7fd1c8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:27.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81036e0 0x7fd1c8198960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800e9c0 tx=0x7fd1b800ee80 comp rx=0 tx=0).stop 2024-09-11T22:12:27.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 shutdown_connections 2024-09-11T22:12:27.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd1b40a2b80 0x7fd1b40a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1c81082a0 0x7fd1c819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c81036e0 0x7fd1c8198960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 --2- 172.21.15.92:0/824986867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1c8102d30 0x7fd1c8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:27.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 >> 172.21.15.92:0/824986867 conn(0x7fd1c80fc7d0 msgr2=0x7fd1c8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:27.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 shutdown_connections 2024-09-11T22:12:27.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:27.450+0000 7fd1cf288640 1 -- 172.21.15.92:0/824986867 wait complete. 2024-09-11T22:12:28.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:27 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/824986867' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:12:28.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:27 smithi092 ceph-mon[31173]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:28.175 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:12:28.175 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:12:28.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T22:12:28.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:27 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/824986867' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:12:28.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:27 smithi110 ceph-mon[34830]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:28.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:27 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/824986867' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:12:28.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:27 smithi153 ceph-mon[35045]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:28.481 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:28.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 -- 172.21.15.92:0/3193341133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4101df0 msgr2=0x7f65c41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 --2- 172.21.15.92:0/3193341133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4101df0 0x7f65c41021d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009900 tx=0x7f65b402a410 comp rx=0 tx=0).stop 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 -- 172.21.15.92:0/3193341133 shutdown_connections 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 --2- 172.21.15.92:0/3193341133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65c4106f20 0x7f65c410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 --2- 172.21.15.92:0/3193341133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c41027a0 0x7f65c41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 --2- 172.21.15.92:0/3193341133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4101df0 0x7f65c41021d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 -- 172.21.15.92:0/3193341133 >> 172.21.15.92:0/3193341133 conn(0x7f65c40fd8b0 msgr2=0x7f65c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 -- 172.21.15.92:0/3193341133 shutdown_connections 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 -- 172.21.15.92:0/3193341133 wait complete. 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.979+0000 7f65ca65c640 1 Processor -- start 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65ca65c640 1 -- start start 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65ca65c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c4101df0 0x7f65c4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65ca65c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65c41027a0 0x7f65c41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65ca65c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4106f20 0x7f65c419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65ca65c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4111e50 con 0x7f65c4106f20 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65ca65c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4111cd0 con 0x7f65c41027a0 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65ca65c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4111fd0 con 0x7f65c4101df0 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4106f20 0x7f65c419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4106f20 0x7f65c419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51240/0 (socket says 172.21.15.92:51240) 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 -- 172.21.15.92:0/1313945582 learned_addr learned my addr 172.21.15.92:0/1313945582 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c3fff640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c4101df0 0x7f65c4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c37fe640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65c41027a0 0x7f65c41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 -- 172.21.15.92:0/1313945582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c4101df0 msgr2=0x7f65c4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c4101df0 0x7f65c4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 -- 172.21.15.92:0/1313945582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65c41027a0 msgr2=0x7f65c41988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65c41027a0 0x7f65c41988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:28.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.980+0000 7f65c8bf9640 1 -- 172.21.15.92:0/1313945582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c419d340 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.981+0000 7f65c8bf9640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4106f20 0x7f65c419cc60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00cce0 tx=0x7f65bc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.981+0000 7f65c17fa640 1 -- 172.21.15.92:0/1313945582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00f450 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.981+0000 7f65c17fa640 1 -- 172.21.15.92:0/1313945582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65bc00faa0 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.981+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c419d630 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.981+0000 7f65c17fa640 1 -- 172.21.15.92:0/1313945582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc018b70 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.981+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a4f10 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.982+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c41021d0 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.983+0000 7f65c17fa640 1 -- 172.21.15.92:0/1313945582 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f65bc018d10 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.983+0000 7f65c17fa640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65ac0a2b80 0x7f65ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.983+0000 7f65c3fff640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65ac0a2b80 0x7f65ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.984+0000 7f65c17fa640 1 -- 172.21.15.92:0/1313945582 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f65bc0cf7c0 con 0x7f65c4106f20 2024-09-11T22:12:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.984+0000 7f65c3fff640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65ac0a2b80 0x7f65ac0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65b4031440 tx=0x7f65b40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:28.987+0000 7f65c17fa640 1 -- 172.21.15.92:0/1313945582 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65bc014030 con 0x7f65c4106f20 2024-09-11T22:12:29.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.165+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65c41996f0 con 0x7f65c4106f20 2024-09-11T22:12:29.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.165+0000 7f65c17fa640 1 -- 172.21.15.92:0/1313945582 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+589 (secure 0 0 0) 0x7f65bc09c4e0 con 0x7f65c4106f20 2024-09-11T22:12:29.167 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2458669467 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:29.167 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2872938890 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:29.167 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3563316301 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:29.167 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:29.167 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3468435378 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:29.167 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/125981315 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:29.168 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:29.168 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:29.168 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:12:29.168 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/4186575934 2024-09-12T22:04:52.791314+0000 2024-09-11T22:12:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:listed 10 entries 2024-09-11T22:12:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.167+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65ac0a2b80 msgr2=0x7f65ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.167+0000 7f65ca65c640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65ac0a2b80 0x7f65ac0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65b4031440 tx=0x7f65b40095f0 comp rx=0 tx=0).stop 2024-09-11T22:12:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.167+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4106f20 msgr2=0x7f65c419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.167+0000 7f65ca65c640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4106f20 0x7f65c419cc60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00cce0 tx=0x7f65bc00a620 comp rx=0 tx=0).stop 2024-09-11T22:12:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 shutdown_connections 2024-09-11T22:12:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65ac0a2b80 0x7f65ac0a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65c4106f20 0x7f65c419cc60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65c41027a0 0x7f65c41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 --2- 172.21.15.92:0/1313945582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c4101df0 0x7f65c4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 >> 172.21.15.92:0/1313945582 conn(0x7f65c40fd8b0 msgr2=0x7f65c40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 shutdown_connections 2024-09-11T22:12:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:29.168+0000 7f65ca65c640 1 -- 172.21.15.92:0/1313945582 wait complete. 2024-09-11T22:12:29.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:29 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1313945582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:12:29.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:29 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1313945582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:12:29.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:29 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1313945582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:12:29.867 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2024-09-11T22:12:29.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:29.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-11T22:12:29.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[76140]: fuse finished with error 0 and tester_r 0 2024-09-11T22:12:29.895 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T22:12:29.913 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T22:12:29.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:12:29.913 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-11T22:12:29.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:12:30.260 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:30.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:30 smithi092 ceph-mon[31173]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:30.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:30 smithi153 ceph-mon[35045]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/59121959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98102570 msgr2=0x7f8e981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/59121959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98102570 0x7f8e981029b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009900 tx=0x7f8e8002a480 comp rx=0 tx=0).stop 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/59121959 shutdown_connections 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/59121959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e98102ef0 0x7f8e9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/59121959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98102570 0x7f8e981029b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/59121959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e98100e20 0x7f8e98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/59121959 >> 172.21.15.92:0/59121959 conn(0x7f8e980fc7d0 msgr2=0x7f8e980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/59121959 shutdown_connections 2024-09-11T22:12:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/59121959 wait complete. 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 Processor -- start 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.690+0000 7f8e9e8dc640 1 -- start start 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98100e20 0x7f8e981984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e98102570 0x7f8e98198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e98102ef0 0x7f8e9819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e98111fd0 con 0x7f8e98100e20 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e98111e50 con 0x7f8e98102570 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e98112150 con 0x7f8e98102ef0 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98100e20 0x7f8e981984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98100e20 0x7f8e981984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51264/0 (socket says 172.21.15.92:51264) 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 -- 172.21.15.92:0/645366164 learned_addr learned my addr 172.21.15.92:0/645366164 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9ce79640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e98102ef0 0x7f8e9819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 -- 172.21.15.92:0/645366164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e98102ef0 msgr2=0x7f8e9819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e98102ef0 0x7f8e9819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 -- 172.21.15.92:0/645366164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e98102570 msgr2=0x7f8e98198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e98102570 0x7f8e98198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 -- 172.21.15.92:0/645366164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9819d480 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9ce79640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e98102ef0 0x7f8e9819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e97fff640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98100e20 0x7f8e981984d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b4a0 tx=0x7f8e8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e95ffb640 1 -- 172.21.15.92:0/645366164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88014070 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9819d770 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e95ffb640 1 -- 172.21.15.92:0/645366164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e88010ac0 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e95ffb640 1 -- 172.21.15.92:0/645366164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88017c30 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.691+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e981a5050 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.692+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e98101200 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.693+0000 7f8e95ffb640 1 -- 172.21.15.92:0/645366164 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8e88017070 con 0x7f8e98100e20 2024-09-11T22:12:30.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.694+0000 7f8e95ffb640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e840a2b80 0x7f8e840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.694+0000 7f8e8ffff640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e840a2b80 0x7f8e840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.694+0000 7f8e95ffb640 1 -- 172.21.15.92:0/645366164 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8e880cea10 con 0x7f8e98100e20 2024-09-11T22:12:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.695+0000 7f8e8ffff640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e840a2b80 0x7f8e840a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e8002a9a0 tx=0x7f8e80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.698+0000 7f8e95ffb640 1 -- 172.21.15.92:0/645366164 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e8809b730 con 0x7f8e98100e20 2024-09-11T22:12:30.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:30 smithi110 ceph-mon[34830]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:12:30.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.865+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e98109090 con 0x7f8e98100e20 2024-09-11T22:12:30.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.866+0000 7f8e95ffb640 1 -- 172.21.15.92:0/645366164 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7f8e8809b030 con 0x7f8e98100e20 2024-09-11T22:12:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:30.868 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e840a2b80 msgr2=0x7f8e840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e840a2b80 0x7f8e840a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e8002a9a0 tx=0x7f8e80000bc0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98100e20 msgr2=0x7f8e981984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98100e20 0x7f8e981984d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b4a0 tx=0x7f8e8800b960 comp rx=0 tx=0).stop 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 shutdown_connections 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e840a2b80 0x7f8e840a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e98102ef0 0x7f8e9819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e98102570 0x7f8e98198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 --2- 172.21.15.92:0/645366164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e98100e20 0x7f8e981984d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.868+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 >> 172.21.15.92:0/645366164 conn(0x7f8e980fc7d0 msgr2=0x7f8e980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.869+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 shutdown_connections 2024-09-11T22:12:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:30.869+0000 7f8e9e8dc640 1 -- 172.21.15.92:0/645366164 wait complete. 2024-09-11T22:12:31.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:31 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/645366164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:31.499 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:12:31.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:31 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/645366164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:31.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:31 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/645366164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:31.741 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:32.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.202+0000 7f092cf5c640 1 -- 172.21.15.92:0/2208408947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0928102da0 msgr2=0x7f0928103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:32.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.202+0000 7f092cf5c640 1 --2- 172.21.15.92:0/2208408947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0928102da0 0x7f0928103180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0920009ca0 tx=0x7f092002a950 comp rx=0 tx=0).stop 2024-09-11T22:12:32.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 -- 172.21.15.92:0/2208408947 shutdown_connections 2024-09-11T22:12:32.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 --2- 172.21.15.92:0/2208408947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f092810c180 0x7f092810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 --2- 172.21.15.92:0/2208408947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09281036c0 0x7f092810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 --2- 172.21.15.92:0/2208408947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0928102da0 0x7f0928103180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 -- 172.21.15.92:0/2208408947 >> 172.21.15.92:0/2208408947 conn(0x7f09280fc7a0 msgr2=0x7f09280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 -- 172.21.15.92:0/2208408947 shutdown_connections 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 -- 172.21.15.92:0/2208408947 wait complete. 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 Processor -- start 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.203+0000 7f092cf5c640 1 -- start start 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0928102da0 0x7f09281983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09281036c0 0x7f0928198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f092810c180 0x7f092819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0928111e50 con 0x7f09281036c0 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0928111cd0 con 0x7f092810c180 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0928111fd0 con 0x7f0928102da0 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09281036c0 0x7f0928198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09281036c0 0x7f0928198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51284/0 (socket says 172.21.15.92:51284) 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 -- 172.21.15.92:0/1952208720 learned_addr learned my addr 172.21.15.92:0/1952208720 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092659c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0928102da0 0x7f09281983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 -- 172.21.15.92:0/1952208720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0928102da0 msgr2=0x7f09281983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0928102da0 0x7f09281983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 -- 172.21.15.92:0/1952208720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f092810c180 msgr2=0x7f092819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0926d9d640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f092810c180 0x7f092819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f092810c180 0x7f092819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 -- 172.21.15.92:0/1952208720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092819d3a0 con 0x7f09281036c0 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0926d9d640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f092810c180 0x7f092819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f0925d9b640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09281036c0 0x7f0928198930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f091000d950 tx=0x7f091000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f091f7fe640 1 -- 172.21.15.92:0/1952208720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910017030 con 0x7f09281036c0 2024-09-11T22:12:32.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f091f7fe640 1 -- 172.21.15.92:0/1952208720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0910004020 con 0x7f09281036c0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092819d690 con 0x7f09281036c0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f091f7fe640 1 -- 172.21.15.92:0/1952208720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100047a0 con 0x7f09281036c0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.204+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09281a4f70 con 0x7f09281036c0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.205+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0928103180 con 0x7f09281036c0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.206+0000 7f091f7fe640 1 -- 172.21.15.92:0/1952208720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f09100041d0 con 0x7f09281036c0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.207+0000 7f091f7fe640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09140a2b80 0x7f09140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.207+0000 7f092659c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09140a2b80 0x7f09140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.207+0000 7f091f7fe640 1 -- 172.21.15.92:0/1952208720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f09100cedc0 con 0x7f09281036c0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.208+0000 7f092659c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09140a2b80 0x7f09140a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0920005fc0 tx=0x7f0920009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.211+0000 7f091f7fe640 1 -- 172.21.15.92:0/1952208720 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f091009bae0 con 0x7f09281036c0 2024-09-11T22:12:32.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.383+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0928199750 con 0x7f09281036c0 2024-09-11T22:12:32.385 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:32 smithi092 ceph-mon[31173]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T22:12:32.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.385+0000 7f091f7fe640 1 -- 172.21.15.92:0/1952208720 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7f0928199750 con 0x7f09281036c0 2024-09-11T22:12:32.386 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:32.386 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09140a2b80 msgr2=0x7f09140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09140a2b80 0x7f09140a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0920005fc0 tx=0x7f0920009360 comp rx=0 tx=0).stop 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09281036c0 msgr2=0x7f0928198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09281036c0 0x7f0928198930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f091000d950 tx=0x7f091000de10 comp rx=0 tx=0).stop 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 shutdown_connections 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09140a2b80 0x7f09140a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f092810c180 0x7f092819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09281036c0 0x7f0928198930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 --2- 172.21.15.92:0/1952208720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0928102da0 0x7f09281983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.387+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 >> 172.21.15.92:0/1952208720 conn(0x7f09280fc7a0 msgr2=0x7f09280fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.388+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 shutdown_connections 2024-09-11T22:12:32.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:32.388+0000 7f092cf5c640 1 -- 172.21.15.92:0/1952208720 wait complete. 2024-09-11T22:12:32.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:32 smithi153 ceph-mon[35045]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T22:12:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:32 smithi110 ceph-mon[34830]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T22:12:33.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T22:12:33.258 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:33.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:33 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1952208720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:33.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:33 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1952208720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:33.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:33 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1952208720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 -- 172.21.15.92:0/3481019718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140100a80 msgr2=0x7f2140100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 --2- 172.21.15.92:0/3481019718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140100a80 0x7f2140100ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2128009930 tx=0x7f212802a420 comp rx=0 tx=0).stop 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 -- 172.21.15.92:0/3481019718 shutdown_connections 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 --2- 172.21.15.92:0/3481019718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2140101420 0x7f214010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 --2- 172.21.15.92:0/3481019718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140100a80 0x7f2140100ee0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 --2- 172.21.15.92:0/3481019718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2140106a80 0x7f2140106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 -- 172.21.15.92:0/3481019718 >> 172.21.15.92:0/3481019718 conn(0x7f21400fc7a0 msgr2=0x7f21400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 -- 172.21.15.92:0/3481019718 shutdown_connections 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 -- 172.21.15.92:0/3481019718 wait complete. 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 Processor -- start 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.661+0000 7f2146569640 1 -- start start 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2146569640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2140100a80 0x7f2140199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2146569640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2140101420 0x7f21401997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2146569640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140106a80 0x7f21401a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2146569640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2140110d50 con 0x7f2140106a80 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2146569640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2140110bd0 con 0x7f2140101420 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2146569640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2140110ed0 con 0x7f2140100a80 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140106a80 0x7f21401a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140106a80 0x7f21401a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51298/0 (socket says 172.21.15.92:51298) 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 -- 172.21.15.92:0/2834329965 learned_addr learned my addr 172.21.15.92:0/2834329965 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2137fff640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2140101420 0x7f21401997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f213ffff640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2140100a80 0x7f2140199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 -- 172.21.15.92:0/2834329965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2140100a80 msgr2=0x7f2140199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2140100a80 0x7f2140199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 -- 172.21.15.92:0/2834329965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2140101420 msgr2=0x7f21401997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2140101420 0x7f21401997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 -- 172.21.15.92:0/2834329965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21401a4840 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2144b06640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140106a80 0x7f21401a4300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f213800eaa0 tx=0x7f213800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f213dffb640 1 -- 172.21.15.92:0/2834329965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138004d20 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f213dffb640 1 -- 172.21.15.92:0/2834329965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2138005020 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.662+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21401a4a10 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.663+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21401a4ed0 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.663+0000 7f213dffb640 1 -- 172.21.15.92:0/2834329965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380116b0 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.663+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2120005290 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.665+0000 7f213dffb640 1 -- 172.21.15.92:0/2834329965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f213800caf0 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.665+0000 7f213dffb640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f212c0a2c00 0x7f212c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.666+0000 7f213ffff640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f212c0a2c00 0x7f212c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.666+0000 7f213dffb640 1 -- 172.21.15.92:0/2834329965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f21380cf260 con 0x7f2140106a80 2024-09-11T22:12:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.666+0000 7f213ffff640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f212c0a2c00 0x7f212c0a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2130009990 tx=0x7f2130009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.670+0000 7f213dffb640 1 -- 172.21.15.92:0/2834329965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2138016030 con 0x7f2140106a80 2024-09-11T22:12:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:33.875+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2120005580 con 0x7f2140106a80 2024-09-11T22:12:34.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.241+0000 7f213dffb640 1 -- 172.21.15.92:0/2834329965 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f213809bf80 con 0x7f2140106a80 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f212c0a2c00 msgr2=0x7f212c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f212c0a2c00 0x7f212c0a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2130009990 tx=0x7f2130009110 comp rx=0 tx=0).stop 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140106a80 msgr2=0x7f21401a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140106a80 0x7f21401a4300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f213800eaa0 tx=0x7f213800eea0 comp rx=0 tx=0).stop 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 shutdown_connections 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f212c0a2c00 0x7f212c0a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2140106a80 0x7f21401a4300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:34.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2140101420 0x7f21401997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:34.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 --2- 172.21.15.92:0/2834329965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2140100a80 0x7f2140199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:34.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 >> 172.21.15.92:0/2834329965 conn(0x7f21400fc7a0 msgr2=0x7f21400fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:34.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 shutdown_connections 2024-09-11T22:12:34.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:34.244+0000 7f2146569640 1 -- 172.21.15.92:0/2834329965 wait complete. 2024-09-11T22:12:34.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:34 smithi092 ceph-mon[31173]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T22:12:34.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:34 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2834329965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:12:34.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:34 smithi153 ceph-mon[35045]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T22:12:34.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:34 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2834329965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:12:34.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:34 smithi110 ceph-mon[34830]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T22:12:34.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:34 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2834329965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:12:34.968 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-11T22:12:34.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T22:12:35.216 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:35.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:35 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2834329965' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:12:35.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:35 smithi092 ceph-mon[31173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:35.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:35 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2834329965' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:12:35.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:35 smithi153 ceph-mon[35045]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.656+0000 7ff82d6c2640 1 -- 172.21.15.92:0/2762620023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828103780 msgr2=0x7ff828107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.656+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/2762620023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828103780 0x7ff828107bc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff810009930 tx=0x7ff81002a420 comp rx=0 tx=0).stop 2024-09-11T22:12:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.656+0000 7ff82d6c2640 1 -- 172.21.15.92:0/2762620023 shutdown_connections 2024-09-11T22:12:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.656+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/2762620023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828108300 0x7ff82810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.656+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/2762620023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828103780 0x7ff828107bc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.656+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/2762620023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff828102dd0 0x7ff8281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.656+0000 7ff82d6c2640 1 -- 172.21.15.92:0/2762620023 >> 172.21.15.92:0/2762620023 conn(0x7ff8280fc7d0 msgr2=0x7ff8280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.657+0000 7ff82d6c2640 1 -- 172.21.15.92:0/2762620023 shutdown_connections 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.657+0000 7ff82d6c2640 1 -- 172.21.15.92:0/2762620023 wait complete. 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.657+0000 7ff82d6c2640 1 Processor -- start 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.657+0000 7ff82d6c2640 1 -- start start 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.657+0000 7ff82d6c2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff828102dd0 0x7ff828198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.657+0000 7ff82d6c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828103780 0x7ff828198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff82d6c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828108300 0x7ff82819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff82d6c2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff828112000 con 0x7ff828108300 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff82d6c2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff828111e80 con 0x7ff828102dd0 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff82d6c2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff828112180 con 0x7ff828103780 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828108300 0x7ff82819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828108300 0x7ff82819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43808/0 (socket says 172.21.15.92:43808) 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 -- 172.21.15.92:0/150655565 learned_addr learned my addr 172.21.15.92:0/150655565 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8267fc640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828103780 0x7ff828198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff826ffd640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff828102dd0 0x7ff828198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 -- 172.21.15.92:0/150655565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828103780 msgr2=0x7ff828198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828103780 0x7ff828198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 -- 172.21.15.92:0/150655565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff828102dd0 msgr2=0x7ff828198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff828102dd0 0x7ff828198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 -- 172.21.15.92:0/150655565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82819d5f0 con 0x7ff828108300 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff826ffd640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff828102dd0 0x7ff828198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff8277fe640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828108300 0x7ff82819cf10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fd0 tx=0x7ff82000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff81ffff640 1 -- 172.21.15.92:0/150655565 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820011830 con 0x7ff828108300 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82819d8e0 con 0x7ff828108300 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff81ffff640 1 -- 172.21.15.92:0/150655565 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff820016070 con 0x7ff828108300 2024-09-11T22:12:35.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff81ffff640 1 -- 172.21.15.92:0/150655565 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820010de0 con 0x7ff828108300 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.658+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a51c0 con 0x7ff828108300 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.659+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff828103230 con 0x7ff828108300 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.661+0000 7ff81ffff640 1 -- 172.21.15.92:0/150655565 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8200119d0 con 0x7ff828108300 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.662+0000 7ff81ffff640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff8140a2b80 0x7ff8140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.662+0000 7ff826ffd640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff8140a2b80 0x7ff8140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.662+0000 7ff81ffff640 1 -- 172.21.15.92:0/150655565 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff8200cf0d0 con 0x7ff828108300 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.662+0000 7ff826ffd640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff8140a2b80 0x7ff8140a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff818007eb0 tx=0x7ff8180073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:35.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.665+0000 7ff81ffff640 1 -- 172.21.15.92:0/150655565 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff82009bdf0 con 0x7ff828108300 2024-09-11T22:12:35.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:35 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2834329965' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:12:35.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:35 smithi110 ceph-mon[34830]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:12:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:35.869+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff82810a290 con 0x7ff828108300 2024-09-11T22:12:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.116+0000 7ff81ffff640 1 -- 172.21.15.92:0/150655565 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ff820086100 con 0x7ff828108300 2024-09-11T22:12:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff8140a2b80 msgr2=0x7ff8140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff8140a2b80 0x7ff8140a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff818007eb0 tx=0x7ff8180073c0 comp rx=0 tx=0).stop 2024-09-11T22:12:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828108300 msgr2=0x7ff82819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828108300 0x7ff82819cf10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fd0 tx=0x7ff82000b8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 shutdown_connections 2024-09-11T22:12:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff8140a2b80 0x7ff8140a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff828108300 0x7ff82819cf10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828103780 0x7ff828198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 --2- 172.21.15.92:0/150655565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff828102dd0 0x7ff828198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 >> 172.21.15.92:0/150655565 conn(0x7ff8280fc7d0 msgr2=0x7ff828105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 shutdown_connections 2024-09-11T22:12:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:36.118+0000 7ff82d6c2640 1 -- 172.21.15.92:0/150655565 wait complete. 2024-09-11T22:12:36.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:36 smithi092 ceph-mon[31173]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T22:12:36.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:36 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/150655565' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:12:36.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:36 smithi092 ceph-mon[31173]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:12:36.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:36 smithi092 ceph-mon[31173]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:12:36.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:36 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/150655565' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:12:36.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:36 smithi092 ceph-mon[31173]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T22:12:36.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:36 smithi092 ceph-mon[31173]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T22:12:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:36 smithi153 ceph-mon[35045]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T22:12:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:36 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/150655565' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:12:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:36 smithi153 ceph-mon[35045]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:12:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:36 smithi153 ceph-mon[35045]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:12:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:36 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/150655565' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:12:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:36 smithi153 ceph-mon[35045]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T22:12:36.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:36 smithi153 ceph-mon[35045]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T22:12:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:36 smithi110 ceph-mon[34830]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T22:12:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:36 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/150655565' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:12:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:36 smithi110 ceph-mon[34830]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:12:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:36 smithi110 ceph-mon[34830]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:12:36.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:36 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/150655565' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:12:36.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:36 smithi110 ceph-mon[34830]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T22:12:36.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:36 smithi110 ceph-mon[34830]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T22:12:36.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T22:12:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:37.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.623+0000 7f612b2bd640 1 -- 172.21.15.92:0/3093718048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124101420 msgr2=0x7f612410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:37.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.623+0000 7f612b2bd640 1 --2- 172.21.15.92:0/3093718048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124101420 0x7f612410d950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f611c009cd0 tx=0x7f611c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 -- 172.21.15.92:0/3093718048 shutdown_connections 2024-09-11T22:12:37.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 --2- 172.21.15.92:0/3093718048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124101420 0x7f612410d950 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 --2- 172.21.15.92:0/3093718048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6124100a80 0x7f6124100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 --2- 172.21.15.92:0/3093718048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6124106a80 0x7f6124106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 -- 172.21.15.92:0/3093718048 >> 172.21.15.92:0/3093718048 conn(0x7f61240fc7a0 msgr2=0x7f61240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 -- 172.21.15.92:0/3093718048 shutdown_connections 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 -- 172.21.15.92:0/3093718048 wait complete. 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 Processor -- start 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 -- start start 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124100a80 0x7f61241983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6124101420 0x7f61241988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.624+0000 7f612b2bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6124106a80 0x7f612419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f612b2bd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6124110d50 con 0x7f6124100a80 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f612b2bd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6124110bd0 con 0x7f6124101420 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f612b2bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6124110ed0 con 0x7f6124106a80 2024-09-11T22:12:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124100a80 0x7f61241983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124100a80 0x7f61241983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43822/0 (socket says 172.21.15.92:43822) 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 -- 172.21.15.92:0/322964401 learned_addr learned my addr 172.21.15.92:0/322964401 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6128858640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6124101420 0x7f61241988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 -- 172.21.15.92:0/322964401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6124106a80 msgr2=0x7f612419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6124106a80 0x7f612419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 -- 172.21.15.92:0/322964401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6124101420 msgr2=0x7f61241988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6124101420 0x7f61241988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 -- 172.21.15.92:0/322964401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612419d350 con 0x7f6124100a80 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f6129059640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124100a80 0x7f61241983a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f612000b4a0 tx=0x7f612000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f611a7fc640 1 -- 172.21.15.92:0/322964401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120014070 con 0x7f6124100a80 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f611a7fc640 1 -- 172.21.15.92:0/322964401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6120010ac0 con 0x7f6124100a80 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419d640 con 0x7f6124100a80 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f611a7fc640 1 -- 172.21.15.92:0/322964401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120017c30 con 0x7f6124100a80 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.625+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a4f20 con 0x7f6124100a80 2024-09-11T22:12:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.626+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6124100ee0 con 0x7f6124100a80 2024-09-11T22:12:37.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.627+0000 7f611a7fc640 1 -- 172.21.15.92:0/322964401 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6120017070 con 0x7f6124100a80 2024-09-11T22:12:37.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.628+0000 7f611a7fc640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f61100a2b80 0x7f61100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:37.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.628+0000 7f6128858640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f61100a2b80 0x7f61100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:37.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.628+0000 7f611a7fc640 1 -- 172.21.15.92:0/322964401 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f61200ceae0 con 0x7f6124100a80 2024-09-11T22:12:37.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.629+0000 7f6128858640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f61100a2b80 0x7f61100a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61240ffde0 tx=0x7f611400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:37.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.631+0000 7f611a7fc640 1 -- 172.21.15.92:0/322964401 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f612009b800 con 0x7f6124100a80 2024-09-11T22:12:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:37.835+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f61241a5150 con 0x7f6124100a80 2024-09-11T22:12:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.126+0000 7f611a7fc640 1 -- 172.21.15.92:0/322964401 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f612009b100 con 0x7f6124100a80 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.128+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f61100a2b80 msgr2=0x7f61100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.128+0000 7f612b2bd640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f61100a2b80 0x7f61100a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61240ffde0 tx=0x7f611400a4c0 comp rx=0 tx=0).stop 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.128+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124100a80 msgr2=0x7f61241983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.128+0000 7f612b2bd640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124100a80 0x7f61241983a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f612000b4a0 tx=0x7f612000b960 comp rx=0 tx=0).stop 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 shutdown_connections 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f61100a2b80 0x7f61100a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6124106a80 0x7f612419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6124101420 0x7f61241988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 --2- 172.21.15.92:0/322964401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6124100a80 0x7f61241983a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 >> 172.21.15.92:0/322964401 conn(0x7f61240fc7a0 msgr2=0x7f612406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 shutdown_connections 2024-09-11T22:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:38.129+0000 7f612b2bd640 1 -- 172.21.15.92:0/322964401 wait complete. 2024-09-11T22:12:38.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:38 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] up:active 2024-09-11T22:12:38.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:38 smithi092 ceph-mon[31173]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:12:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:38 smithi092 ceph-mon[31173]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T22:12:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:38 smithi092 ceph-mon[31173]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T22:12:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:38 smithi092 ceph-mon[31173]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T22:12:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:38 smithi092 ceph-mon[31173]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T22:12:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:38 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/322964401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:12:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] up:active 2024-09-11T22:12:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-mon[34830]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:12:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-mon[34830]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T22:12:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-mon[34830]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T22:12:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-mon[34830]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T22:12:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-mon[34830]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T22:12:38.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/322964401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:12:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:38 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] up:active 2024-09-11T22:12:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:38 smithi153 ceph-mon[35045]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:12:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:38 smithi153 ceph-mon[35045]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T22:12:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:38 smithi153 ceph-mon[35045]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T22:12:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:38 smithi153 ceph-mon[35045]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T22:12:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:38 smithi153 ceph-mon[35045]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T22:12:38.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:38 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/322964401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:12:38.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T22:12:39.105 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:39.129 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: 2024-09-11T22:12:38.759+0000 7f818248d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T22:12:39.129 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:12:38 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: 2024-09-11T22:12:38.759+0000 7f818248d640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-11T22:12:39.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:39 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/322964401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:12:39.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:39 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] up:active 2024-09-11T22:12:39.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:39 smithi092 ceph-mon[31173]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:12:39.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:39 smithi092 ceph-mon[31173]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T22:12:39.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:39 smithi092 ceph-mon[31173]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-11T22:12:39.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:39 smithi092 ceph-mon[31173]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-11T22:12:39.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:39 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/322964401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:12:39.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:39 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] up:active 2024-09-11T22:12:39.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:39 smithi110 ceph-mon[34830]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:12:39.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:39 smithi110 ceph-mon[34830]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T22:12:39.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:39 smithi110 ceph-mon[34830]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-11T22:12:39.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:39 smithi110 ceph-mon[34830]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-11T22:12:39.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:39 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/322964401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:12:39.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:39 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] up:active 2024-09-11T22:12:39.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:39 smithi153 ceph-mon[35045]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:12:39.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:39 smithi153 ceph-mon[35045]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T22:12:39.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:39 smithi153 ceph-mon[35045]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-11T22:12:39.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:39 smithi153 ceph-mon[35045]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-11T22:12:39.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 -- 172.21.15.92:0/481399785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10100a80 msgr2=0x7fcf10100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:39.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 --2- 172.21.15.92:0/481399785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10100a80 0x7fcf10100ee0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009930 tx=0x7fcef802a420 comp rx=0 tx=0).stop 2024-09-11T22:12:39.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 -- 172.21.15.92:0/481399785 shutdown_connections 2024-09-11T22:12:39.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 --2- 172.21.15.92:0/481399785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf10101420 0x7fcf1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 --2- 172.21.15.92:0/481399785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10100a80 0x7fcf10100ee0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 --2- 172.21.15.92:0/481399785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf10106a80 0x7fcf10106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 -- 172.21.15.92:0/481399785 >> 172.21.15.92:0/481399785 conn(0x7fcf100fc7a0 msgr2=0x7fcf100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 -- 172.21.15.92:0/481399785 shutdown_connections 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 -- 172.21.15.92:0/481399785 wait complete. 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.526+0000 7fcf16a24640 1 Processor -- start 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf16a24640 1 -- start start 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf16a24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf10100a80 0x7fcf10197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf16a24640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf10101420 0x7fcf10197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf16a24640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10106a80 0x7fcf1019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf16a24640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10110d30 con 0x7fcf10106a80 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf16a24640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10110bb0 con 0x7fcf10100a80 2024-09-11T22:12:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf16a24640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10110eb0 con 0x7fcf10101420 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10106a80 0x7fcf1019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10106a80 0x7fcf1019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43830/0 (socket says 172.21.15.92:43830) 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 -- 172.21.15.92:0/152622126 learned_addr learned my addr 172.21.15.92:0/152622126 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf07fff640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf10101420 0x7fcf10197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf0ffff640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf10100a80 0x7fcf10197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 -- 172.21.15.92:0/152622126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf10101420 msgr2=0x7fcf10197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf10101420 0x7fcf10197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 -- 172.21.15.92:0/152622126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf10100a80 msgr2=0x7fcf10197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf10100a80 0x7fcf10197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 -- 172.21.15.92:0/152622126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1019c2c0 con 0x7fcf10106a80 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf07fff640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf10101420 0x7fcf10197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf0ffff640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf10100a80 0x7fcf10197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.527+0000 7fcf14fc1640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10106a80 0x7fcf1019bbe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800cad0 tx=0x7fcf0800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.528+0000 7fcf0dffb640 1 -- 172.21.15.92:0/152622126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800ae20 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.528+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1019c5b0 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.528+0000 7fcf0dffb640 1 -- 172.21.15.92:0/152622126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf08005020 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.528+0000 7fcf0dffb640 1 -- 172.21.15.92:0/152622126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800f6b0 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.528+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf101a3e90 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.529+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf101021c0 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.530+0000 7fcf0dffb640 1 -- 172.21.15.92:0/152622126 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcf080051d0 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.530+0000 7fcf0dffb640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fcefc0a2b80 0x7fcefc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.530+0000 7fcf0ffff640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fcefc0a2b80 0x7fcefc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.531+0000 7fcf0dffb640 1 -- 172.21.15.92:0/152622126 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcf080cf3b0 con 0x7fcf10106a80 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.531+0000 7fcf0ffff640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fcefc0a2b80 0x7fcefc0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005fd0 tx=0x7fcf000094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.534+0000 7fcf0dffb640 1 -- 172.21.15.92:0/152622126 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf08014030 con 0x7fcf10106a80 2024-09-11T22:12:39.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:39.736+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fcf10108ff0 con 0x7fcf10106a80 2024-09-11T22:12:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.143+0000 7fcf0dffb640 1 -- 172.21.15.92:0/152622126 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fcf0809c0d0 con 0x7fcf10106a80 2024-09-11T22:12:40.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fcefc0a2b80 msgr2=0x7fcefc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fcefc0a2b80 0x7fcefc0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005fd0 tx=0x7fcf000094d0 comp rx=0 tx=0).stop 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10106a80 msgr2=0x7fcf1019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10106a80 0x7fcf1019bbe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800cad0 tx=0x7fcf0800cf90 comp rx=0 tx=0).stop 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 shutdown_connections 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fcefc0a2b80 0x7fcefc0a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf10106a80 0x7fcf1019bbe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf10101420 0x7fcf10197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 --2- 172.21.15.92:0/152622126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf10100a80 0x7fcf10197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.145+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 >> 172.21.15.92:0/152622126 conn(0x7fcf100fc7a0 msgr2=0x7fcf100fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.146+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 shutdown_connections 2024-09-11T22:12:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:40.146+0000 7fcf16a24640 1 -- 172.21.15.92:0/152622126 wait complete. 2024-09-11T22:12:40.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] up:active 2024-09-11T22:12:40.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:12:40.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:12:40.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: Cluster is now healthy 2024-09-11T22:12:40.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-11T22:12:40.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T22:12:40.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-11T22:12:40.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: pgmap v302: 129 pgs: 129 active+clean; 585 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-11T22:12:40.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:40 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/152622126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] up:active 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: Cluster is now healthy 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: pgmap v302: 129 pgs: 129 active+clean; 585 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-11T22:12:40.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:40 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/152622126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] up:active 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: Cluster is now healthy 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: pgmap v302: 129 pgs: 129 active+clean; 585 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-11T22:12:40.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:40 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/152622126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:12:40.888 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-11T22:12:40.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T22:12:41.132 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:41.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/152622126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:12:41.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] up:active 2024-09-11T22:12:41.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-mon[31173]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T22:12:41.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:12:41.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:41 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/152622126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:12:41.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:41 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] up:active 2024-09-11T22:12:41.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:41 smithi110 ceph-mon[34830]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T22:12:41.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:41 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:12:41.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:41 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/152622126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:12:41.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:41 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] up:active 2024-09-11T22:12:41.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:41 smithi153 ceph-mon[35045]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T22:12:41.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:41 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.594+0000 7f2c14028640 1 -- 172.21.15.92:0/2219697372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c101de0 msgr2=0x7f2c0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.594+0000 7f2c14028640 1 --2- 172.21.15.92:0/2219697372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c101de0 0x7f2c0c1021c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c0802a410 comp rx=0 tx=0).stop 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.594+0000 7f2c14028640 1 -- 172.21.15.92:0/2219697372 shutdown_connections 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.594+0000 7f2c14028640 1 --2- 172.21.15.92:0/2219697372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c0c106f10 0x7f2c0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.594+0000 7f2c14028640 1 --2- 172.21.15.92:0/2219697372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c0c102790 0x7f2c0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.594+0000 7f2c14028640 1 --2- 172.21.15.92:0/2219697372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c101de0 0x7f2c0c1021c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.594+0000 7f2c14028640 1 -- 172.21.15.92:0/2219697372 >> 172.21.15.92:0/2219697372 conn(0x7f2c0c0fd8c0 msgr2=0x7f2c0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 -- 172.21.15.92:0/2219697372 shutdown_connections 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 -- 172.21.15.92:0/2219697372 wait complete. 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 Processor -- start 2024-09-11T22:12:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 -- start start 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c0c101de0 0x7f2c0c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c0c102790 0x7f2c0c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c106f10 0x7f2c0c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c111e40 con 0x7f2c0c106f10 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c111cc0 con 0x7f2c0c101de0 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c14028640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c111fc0 con 0x7f2c0c102790 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c125c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c106f10 0x7f2c0c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.595+0000 7f2c125c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c106f10 0x7f2c0c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43850/0 (socket says 172.21.15.92:43850) 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c125c5640 1 -- 172.21.15.92:0/1313076075 learned_addr learned my addr 172.21.15.92:0/1313076075 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c11dc4640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c0c101de0 0x7f2c0c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c115c3640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c0c102790 0x7f2c0c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c125c5640 1 -- 172.21.15.92:0/1313076075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c0c102790 msgr2=0x7f2c0c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c125c5640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c0c102790 0x7f2c0c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c125c5640 1 -- 172.21.15.92:0/1313076075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c0c101de0 msgr2=0x7f2c0c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c125c5640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c0c101de0 0x7f2c0c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c125c5640 1 -- 172.21.15.92:0/1313076075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c19d3a0 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c125c5640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c106f10 0x7f2c0c19ccc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400eaa0 tx=0x7f2c0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c02ffd640 1 -- 172.21.15.92:0/1313076075 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0400cdb0 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c19d690 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c02ffd640 1 -- 172.21.15.92:0/1313076075 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c04005020 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c02ffd640 1 -- 172.21.15.92:0/1313076075 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040116d0 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.596+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c1a4f70 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.597+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c1021c0 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.598+0000 7f2c02ffd640 1 -- 172.21.15.92:0/1313076075 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c040051d0 con 0x7f2c0c106f10 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.599+0000 7f2c02ffd640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2bf80a2b80 0x7f2bf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.599+0000 7f2c11dc4640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2bf80a2b80 0x7f2bf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.599+0000 7f2c02ffd640 1 -- 172.21.15.92:0/1313076075 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2c040cf810 con 0x7f2c0c106f10 2024-09-11T22:12:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.599+0000 7f2c11dc4640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2bf80a2b80 0x7f2bf80a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c08031440 tx=0x7f2c08005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.602+0000 7f2c02ffd640 1 -- 172.21.15.92:0/1313076075 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c04016030 con 0x7f2c0c106f10 2024-09-11T22:12:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.793+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2c0c062630 con 0x7f2bf80a2b80 2024-09-11T22:12:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.822+0000 7f2c02ffd640 1 -- 172.21.15.92:0/1313076075 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c0c062630 con 0x7f2bf80a2b80 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.823+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2bf80a2b80 msgr2=0x7f2bf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.823+0000 7f2c14028640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2bf80a2b80 0x7f2bf80a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c08031440 tx=0x7f2c08005b50 comp rx=0 tx=0).stop 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c106f10 msgr2=0x7f2c0c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c106f10 0x7f2c0c19ccc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400eaa0 tx=0x7f2c0400ef60 comp rx=0 tx=0).stop 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 shutdown_connections 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2bf80a2b80 0x7f2bf80a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c106f10 0x7f2c0c19ccc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c0c102790 0x7f2c0c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 --2- 172.21.15.92:0/1313076075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c0c101de0 0x7f2c0c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 >> 172.21.15.92:0/1313076075 conn(0x7f2c0c0fd8c0 msgr2=0x7f2c0c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 shutdown_connections 2024-09-11T22:12:41.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:41.824+0000 7f2c14028640 1 -- 172.21.15.92:0/1313076075 wait complete. 2024-09-11T22:12:42.126 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:12:41.798+0000 7f8e35675640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:12:42.126 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:12:41.798+0000 7f8e35675640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:12:42.126 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:12:41.798+0000 7f8e35675640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:12:42.126 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:12:41.798+0000 7f8e35675640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:12:42.126 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:12:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:12:41.798+0000 7f8e35675640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:12:42.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:42 smithi110 ceph-mon[34830]: pgmap v303: 129 pgs: 129 active+clean; 586 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-09-11T22:12:42.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:42 smithi110 ceph-mon[34830]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:42.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:42 smithi092 ceph-mon[31173]: pgmap v303: 129 pgs: 129 active+clean; 586 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-09-11T22:12:42.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:42 smithi092 ceph-mon[31173]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:42.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:42 smithi153 ceph-mon[35045]: pgmap v303: 129 pgs: 129 active+clean; 586 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-09-11T22:12:42.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:42 smithi153 ceph-mon[35045]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:42.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T22:12:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 -- 172.21.15.92:0/3635339515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc106f50 msgr2=0x7f7ecc10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3635339515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc106f50 0x7f7ecc10eaf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009cd0 tx=0x7f7ec402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 -- 172.21.15.92:0/3635339515 shutdown_connections 2024-09-11T22:12:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3635339515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc106f50 0x7f7ecc10eaf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3635339515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc100ca0 0x7f7ecc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3635339515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ecc1002f0 0x7f7ecc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 -- 172.21.15.92:0/3635339515 >> 172.21.15.92:0/3635339515 conn(0x7f7ecc077210 msgr2=0x7f7ecc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 -- 172.21.15.92:0/3635339515 shutdown_connections 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.290+0000 7f7ed2085640 1 -- 172.21.15.92:0/3635339515 wait complete. 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 Processor -- start 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 -- start start 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc1002f0 0x7f7ecc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc100ca0 0x7f7ecc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ecc106f50 0x7f7ecc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111e50 con 0x7f7ecc100ca0 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111cd0 con 0x7f7ecc106f50 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ed2085640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111fd0 con 0x7f7ecc1002f0 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc100ca0 0x7f7ecc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc100ca0 0x7f7ecc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43896/0 (socket says 172.21.15.92:43896) 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecaffd640 1 -- 172.21.15.92:0/3167684956 learned_addr learned my addr 172.21.15.92:0/3167684956 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecb7fe640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc1002f0 0x7f7ecc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecbfff640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ecc106f50 0x7f7ecc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecaffd640 1 -- 172.21.15.92:0/3167684956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc1002f0 msgr2=0x7f7ecc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecaffd640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc1002f0 0x7f7ecc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.291+0000 7f7ecaffd640 1 -- 172.21.15.92:0/3167684956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ecc106f50 msgr2=0x7f7ecc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ecaffd640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ecc106f50 0x7f7ecc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ecaffd640 1 -- 172.21.15.92:0/3167684956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc19d350 con 0x7f7ecc100ca0 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ecbfff640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ecc106f50 0x7f7ecc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ecaffd640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc100ca0 0x7f7ecc1988e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400d740 tx=0x7f7eb400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ec8ff9640 1 -- 172.21.15.92:0/3167684956 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4017070 con 0x7f7ecc100ca0 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc19d640 con 0x7f7ecc100ca0 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ec8ff9640 1 -- 172.21.15.92:0/3167684956 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb400de50 con 0x7f7ecc100ca0 2024-09-11T22:12:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ec8ff9640 1 -- 172.21.15.92:0/3167684956 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4004d60 con 0x7f7ecc100ca0 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.292+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a4f20 con 0x7f7ecc100ca0 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.293+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc1006d0 con 0x7f7ecc100ca0 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.294+0000 7f7ec8ff9640 1 -- 172.21.15.92:0/3167684956 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7eb4004f00 con 0x7f7ecc100ca0 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.295+0000 7f7ec8ff9640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7eb80a2b80 0x7f7eb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.295+0000 7f7ecb7fe640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7eb80a2b80 0x7f7eb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.295+0000 7f7ec8ff9640 1 -- 172.21.15.92:0/3167684956 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7eb40cebf0 con 0x7f7ecc100ca0 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.295+0000 7f7ecb7fe640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7eb80a2b80 0x7f7eb80a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0058c0 tx=0x7f7ebc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.299+0000 7f7ec8ff9640 1 -- 172.21.15.92:0/3167684956 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7eb409b910 con 0x7f7ecc100ca0 2024-09-11T22:12:43.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.490+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7ecc10a150 con 0x7f7eb80a2b80 2024-09-11T22:12:43.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.515+0000 7f7ec8ff9640 1 -- 172.21.15.92:0/3167684956 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7ecc10a150 con 0x7f7eb80a2b80 2024-09-11T22:12:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7eb80a2b80 msgr2=0x7f7eb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7eb80a2b80 0x7f7eb80a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0058c0 tx=0x7f7ebc00a2d0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc100ca0 msgr2=0x7f7ecc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc100ca0 0x7f7ecc1988e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400d740 tx=0x7f7eb400dc00 comp rx=0 tx=0).stop 2024-09-11T22:12:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 shutdown_connections 2024-09-11T22:12:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7eb80a2b80 0x7f7eb80a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ecc106f50 0x7f7ecc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ecc100ca0 0x7f7ecc1988e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 --2- 172.21.15.92:0/3167684956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc1002f0 0x7f7ecc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 >> 172.21.15.92:0/3167684956 conn(0x7f7ecc077210 msgr2=0x7f7ecc105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 shutdown_connections 2024-09-11T22:12:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:43.517+0000 7f7ed2085640 1 -- 172.21.15.92:0/3167684956 wait complete. 2024-09-11T22:12:44.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T22:12:44.217 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:12:43 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: 2024-09-11T22:12:43.759+0000 7f818248d640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-11T22:12:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:44.958 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:44 smithi092 ceph-mon[31173]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:44.959 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:44 smithi092 ceph-mon[31173]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2024-09-11T22:12:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:44 smithi153 ceph-mon[35045]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:45.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:44 smithi153 ceph-mon[35045]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2024-09-11T22:12:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 -- 172.21.15.92:0/330566679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489810c1a0 msgr2=0x7f489810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 --2- 172.21.15.92:0/330566679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489810c1a0 0x7f489810e610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4890009ca0 tx=0x7f489002a950 comp rx=0 tx=0).stop 2024-09-11T22:12:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 -- 172.21.15.92:0/330566679 shutdown_connections 2024-09-11T22:12:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 --2- 172.21.15.92:0/330566679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489810c1a0 0x7f489810e610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 --2- 172.21.15.92:0/330566679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4898103720 0x7f489810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 --2- 172.21.15.92:0/330566679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4898102e00 0x7f48981031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 -- 172.21.15.92:0/330566679 >> 172.21.15.92:0/330566679 conn(0x7f48980fc7a0 msgr2=0x7f48980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 -- 172.21.15.92:0/330566679 shutdown_connections 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 -- 172.21.15.92:0/330566679 wait complete. 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 Processor -- start 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.041+0000 7f48a010d640 1 -- start start 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f48a010d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4898102e00 0x7f4898198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f48a010d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4898103720 0x7f4898198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f48a010d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f489810c1a0 0x7f489819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f48a010d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4898111de0 con 0x7f4898102e00 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f48a010d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4898111c60 con 0x7f489810c1a0 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f48a010d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4898111f60 con 0x7f4898103720 2024-09-11T22:12:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4898102e00 0x7f4898198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4898102e00 0x7f4898198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43918/0 (socket says 172.21.15.92:43918) 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 -- 172.21.15.92:0/3610088768 learned_addr learned my addr 172.21.15.92:0/3610088768 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489d6a8640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4898103720 0x7f4898198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 -- 172.21.15.92:0/3610088768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4898103720 msgr2=0x7f4898198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4898103720 0x7f4898198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489e6aa640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f489810c1a0 0x7f489819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 -- 172.21.15.92:0/3610088768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f489810c1a0 msgr2=0x7f489819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f489810c1a0 0x7f489819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 -- 172.21.15.92:0/3610088768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489819d2b0 con 0x7f4898102e00 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489d6a8640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4898103720 0x7f4898198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f489dea9640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4898102e00 0x7f4898198300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f489400c890 tx=0x7f489400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f488affd640 1 -- 172.21.15.92:0/3610088768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894004220 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f488affd640 1 -- 172.21.15.92:0/3610088768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48940043c0 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f488affd640 1 -- 172.21.15.92:0/3610088768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894004a00 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.042+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489819d5a0 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.043+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48981a4e80 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.043+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48981031e0 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.045+0000 7f488affd640 1 -- 172.21.15.92:0/3610088768 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4894004ba0 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.045+0000 7f488affd640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f48840a2c90 0x7f48840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.045+0000 7f489d6a8640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f48840a2c90 0x7f48840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.045+0000 7f488affd640 1 -- 172.21.15.92:0/3610088768 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f48940cf590 con 0x7f4898102e00 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.046+0000 7f489d6a8640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f48840a2c90 0x7f48840a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f488c005d90 tx=0x7f488c005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.049+0000 7f488affd640 1 -- 172.21.15.92:0/3610088768 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4894014030 con 0x7f4898102e00 2024-09-11T22:12:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:44 smithi110 ceph-mon[34830]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:44 smithi110 ceph-mon[34830]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2024-09-11T22:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.244+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f489802d830 con 0x7f48840a2c90 2024-09-11T22:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.249+0000 7f488affd640 1 -- 172.21.15.92:0/3610088768 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f489802d830 con 0x7f48840a2c90 2024-09-11T22:12:45.250 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/qa/sv_0/6d5615a6-07ce-4091-80b5-509f10920896 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f48840a2c90 msgr2=0x7f48840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f48840a2c90 0x7f48840a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f488c005d90 tx=0x7f488c005d20 comp rx=0 tx=0).stop 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4898102e00 msgr2=0x7f4898198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4898102e00 0x7f4898198300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f489400c890 tx=0x7f489400cd50 comp rx=0 tx=0).stop 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 shutdown_connections 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f48840a2c90 0x7f48840a5150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f489810c1a0 0x7f489819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4898103720 0x7f4898198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 --2- 172.21.15.92:0/3610088768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4898102e00 0x7f4898198300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 >> 172.21.15.92:0/3610088768 conn(0x7f48980fc7a0 msgr2=0x7f48980fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 shutdown_connections 2024-09-11T22:12:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:45.251+0000 7f48a010d640 1 -- 172.21.15.92:0/3610088768 wait complete. 2024-09-11T22:12:45.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:45 smithi092 ceph-mon[31173]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:12:45.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T22:12:46.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:45 smithi153 ceph-mon[35045]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:12:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:45 smithi110 ceph-mon[34830]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:12:46.231 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:46.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 -- 172.21.15.92:0/3869970778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4101420 msgr2=0x7f6af410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 --2- 172.21.15.92:0/3869970778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4101420 0x7f6af410d950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009cd0 tx=0x7f6aec02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 -- 172.21.15.92:0/3869970778 shutdown_connections 2024-09-11T22:12:46.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 --2- 172.21.15.92:0/3869970778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4101420 0x7f6af410d950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 --2- 172.21.15.92:0/3869970778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af4100a80 0x7f6af4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 --2- 172.21.15.92:0/3869970778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af4106a80 0x7f6af4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 -- 172.21.15.92:0/3869970778 >> 172.21.15.92:0/3869970778 conn(0x7f6af40fc7a0 msgr2=0x7f6af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 -- 172.21.15.92:0/3869970778 shutdown_connections 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.753+0000 7f6afb13e640 1 -- 172.21.15.92:0/3869970778 wait complete. 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 Processor -- start 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 -- start start 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af4100a80 0x7f6af4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af4101420 0x7f6af41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4106a80 0x7f6af419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4110d50 con 0x7f6af4106a80 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4110bd0 con 0x7f6af4101420 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6afb13e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4110ed0 con 0x7f6af4100a80 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af96db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4106a80 0x7f6af419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af96db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4106a80 0x7f6af419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47938/0 (socket says 172.21.15.92:47938) 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af96db640 1 -- 172.21.15.92:0/2854515671 learned_addr learned my addr 172.21.15.92:0/2854515671 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:46.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af3fff640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af4101420 0x7f6af41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af8eda640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af4100a80 0x7f6af4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af96db640 1 -- 172.21.15.92:0/2854515671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af4100a80 msgr2=0x7f6af4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af96db640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af4100a80 0x7f6af4197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af96db640 1 -- 172.21.15.92:0/2854515671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af4101420 msgr2=0x7f6af41977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.754+0000 7f6af96db640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af4101420 0x7f6af41977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.755+0000 7f6af96db640 1 -- 172.21.15.92:0/2854515671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af419c240 con 0x7f6af4106a80 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.755+0000 7f6af96db640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4106a80 0x7f6af419bb60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005cc0 tx=0x7f6aec003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.755+0000 7f6af1ffb640 1 -- 172.21.15.92:0/2854515671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec004450 con 0x7f6af4106a80 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.755+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af419c4d0 con 0x7f6af4106a80 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.755+0000 7f6af1ffb640 1 -- 172.21.15.92:0/2854515671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6aec031c90 con 0x7f6af4106a80 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.755+0000 7f6af1ffb640 1 -- 172.21.15.92:0/2854515671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec03b400 con 0x7f6af4106a80 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.755+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af41a3e10 con 0x7f6af4106a80 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.756+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af4100ee0 con 0x7f6af4106a80 2024-09-11T22:12:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.757+0000 7f6af1ffb640 1 -- 172.21.15.92:0/2854515671 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6aec004680 con 0x7f6af4106a80 2024-09-11T22:12:46.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.758+0000 7f6af1ffb640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae40a2c10 0x7f6ae40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:46.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.758+0000 7f6af1ffb640 1 -- 172.21.15.92:0/2854515671 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6aec02e030 con 0x7f6af4106a80 2024-09-11T22:12:46.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.758+0000 7f6af8eda640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae40a2c10 0x7f6ae40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:46.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.758+0000 7f6af8eda640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae40a2c10 0x7f6ae40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800a810 tx=0x7f6ae8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:46.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.762+0000 7f6af1ffb640 1 -- 172.21.15.92:0/2854515671 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6aec0bd1e0 con 0x7f6af4106a80 2024-09-11T22:12:46.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.954+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6af4062630 con 0x7f6ae40a2c10 2024-09-11T22:12:46.955 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:46 smithi092 ceph-mon[31173]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:46.956 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:46 smithi092 ceph-mon[31173]: pgmap v305: 129 pgs: 129 active+clean; 608 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2024-09-11T22:12:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.979+0000 7f6af1ffb640 1 -- 172.21.15.92:0/2854515671 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6af4062630 con 0x7f6ae40a2c10 2024-09-11T22:12:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae40a2c10 msgr2=0x7f6ae40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae40a2c10 0x7f6ae40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800a810 tx=0x7f6ae8008040 comp rx=0 tx=0).stop 2024-09-11T22:12:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4106a80 msgr2=0x7f6af419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4106a80 0x7f6af419bb60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005cc0 tx=0x7f6aec003c20 comp rx=0 tx=0).stop 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 shutdown_connections 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae40a2c10 0x7f6ae40a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af4106a80 0x7f6af419bb60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af4101420 0x7f6af41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 --2- 172.21.15.92:0/2854515671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af4100a80 0x7f6af4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 >> 172.21.15.92:0/2854515671 conn(0x7f6af40fc7a0 msgr2=0x7f6af40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 shutdown_connections 2024-09-11T22:12:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:46.981+0000 7f6afb13e640 1 -- 172.21.15.92:0/2854515671 wait complete. 2024-09-11T22:12:47.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:46 smithi153 ceph-mon[35045]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:47.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:46 smithi153 ceph-mon[35045]: pgmap v305: 129 pgs: 129 active+clean; 608 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2024-09-11T22:12:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:46 smithi110 ceph-mon[34830]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:46 smithi110 ceph-mon[34830]: pgmap v305: 129 pgs: 129 active+clean; 608 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2024-09-11T22:12:47.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T22:12:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 -- 172.21.15.92:0/2792938686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60101df0 msgr2=0x7fda601021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 --2- 172.21.15.92:0/2792938686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60101df0 0x7fda601021d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda50009900 tx=0x7fda5002a410 comp rx=0 tx=0).stop 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 -- 172.21.15.92:0/2792938686 shutdown_connections 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 --2- 172.21.15.92:0/2792938686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda60106f20 0x7fda6010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 --2- 172.21.15.92:0/2792938686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda601027a0 0x7fda601067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 --2- 172.21.15.92:0/2792938686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60101df0 0x7fda601021d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 -- 172.21.15.92:0/2792938686 >> 172.21.15.92:0/2792938686 conn(0x7fda600fd8f0 msgr2=0x7fda600ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 -- 172.21.15.92:0/2792938686 shutdown_connections 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 -- 172.21.15.92:0/2792938686 wait complete. 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 Processor -- start 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.456+0000 7fda669a7640 1 -- start start 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda669a7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60101df0 0x7fda6019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda669a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda601027a0 0x7fda6019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda669a7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60106f20 0x7fda6019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda669a7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda60111ed0 con 0x7fda60106f20 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda669a7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fda60111d50 con 0x7fda601027a0 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda669a7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda60112050 con 0x7fda60101df0 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60106f20 0x7fda6019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60106f20 0x7fda6019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47952/0 (socket says 172.21.15.92:47952) 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 -- 172.21.15.92:0/3042480710 learned_addr learned my addr 172.21.15.92:0/3042480710 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda5ffff640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60101df0 0x7fda6019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda5f7fe640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda601027a0 0x7fda6019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 -- 172.21.15.92:0/3042480710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60101df0 msgr2=0x7fda6019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60101df0 0x7fda6019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 -- 172.21.15.92:0/3042480710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda601027a0 msgr2=0x7fda6019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda601027a0 0x7fda6019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 -- 172.21.15.92:0/3042480710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6019f620 con 0x7fda60106f20 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.457+0000 7fda64f44640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60106f20 0x7fda6019ef40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda5800b970 tx=0x7fda5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.458+0000 7fda5d7fa640 1 -- 172.21.15.92:0/3042480710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58019070 con 0x7fda60106f20 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.458+0000 7fda5d7fa640 1 -- 172.21.15.92:0/3042480710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda58014440 con 0x7fda60106f20 2024-09-11T22:12:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.458+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6019f910 con 0x7fda60106f20 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.458+0000 7fda5d7fa640 1 -- 172.21.15.92:0/3042480710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda580075d0 con 0x7fda60106f20 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.458+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601a71f0 con 0x7fda60106f20 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.459+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda60102250 con 0x7fda60106f20 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.460+0000 7fda5d7fa640 1 -- 172.21.15.92:0/3042480710 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda5801c070 con 0x7fda60106f20 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.460+0000 7fda5d7fa640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda480a2c10 0x7fda480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.460+0000 7fda5ffff640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda480a2c10 0x7fda480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.461+0000 7fda5d7fa640 1 -- 172.21.15.92:0/3042480710 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fda580ced80 con 0x7fda60106f20 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.461+0000 7fda5ffff640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda480a2c10 0x7fda480a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda50031440 tx=0x7fda50005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.464+0000 7fda5d7fa640 1 -- 172.21.15.92:0/3042480710 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda5809b400 con 0x7fda60106f20 2024-09-11T22:12:48.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.655+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fda60075430 con 0x7fda480a2c10 2024-09-11T22:12:48.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.659+0000 7fda5d7fa640 1 -- 172.21.15.92:0/3042480710 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fda60075430 con 0x7fda480a2c10 2024-09-11T22:12:48.660 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/qa/sv_1/074e8622-570a-4214-ad04-746bf72e077d 2024-09-11T22:12:48.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda480a2c10 msgr2=0x7fda480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda480a2c10 0x7fda480a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda50031440 tx=0x7fda50005e80 comp rx=0 tx=0).stop 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60106f20 msgr2=0x7fda6019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60106f20 0x7fda6019ef40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda5800b970 tx=0x7fda5800be30 comp rx=0 tx=0).stop 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 shutdown_connections 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda480a2c10 0x7fda480a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda60106f20 0x7fda6019ef40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda601027a0 0x7fda6019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 --2- 172.21.15.92:0/3042480710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60101df0 0x7fda6019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 >> 172.21.15.92:0/3042480710 conn(0x7fda600fd8f0 msgr2=0x7fda600fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 shutdown_connections 2024-09-11T22:12:48.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:48.661+0000 7fda669a7640 1 -- 172.21.15.92:0/3042480710 wait complete. 2024-09-11T22:12:49.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:48 smithi092 ceph-mon[31173]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:49.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:48 smithi092 ceph-mon[31173]: pgmap v306: 129 pgs: 129 active+clean; 608 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-11T22:12:49.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:48 smithi153 ceph-mon[35045]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:48 smithi153 ceph-mon[35045]: pgmap v306: 129 pgs: 129 active+clean; 608 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-11T22:12:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:48 smithi110 ceph-mon[34830]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:48 smithi110 ceph-mon[34830]: pgmap v306: 129 pgs: 129 active+clean; 608 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-11T22:12:49.394 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:12:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:50.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:49 smithi092 ceph-mon[31173]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:50.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:49 smithi153 ceph-mon[35045]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:50.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 -- 172.21.15.92:0/1965109219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80106f10 msgr2=0x7f8e8010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:50.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 --2- 172.21.15.92:0/1965109219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80106f10 0x7f8e8010ea90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009ca0 tx=0x7f8e7802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 -- 172.21.15.92:0/1965109219 shutdown_connections 2024-09-11T22:12:50.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 --2- 172.21.15.92:0/1965109219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80106f10 0x7f8e8010ea90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 --2- 172.21.15.92:0/1965109219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e80102790 0x7f8e801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 --2- 172.21.15.92:0/1965109219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e80101de0 0x7f8e801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 -- 172.21.15.92:0/1965109219 >> 172.21.15.92:0/1965109219 conn(0x7f8e800fd8c0 msgr2=0x7f8e800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 -- 172.21.15.92:0/1965109219 shutdown_connections 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 -- 172.21.15.92:0/1965109219 wait complete. 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 Processor -- start 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.174+0000 7f8e85978640 1 -- start start 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e85978640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80101de0 0x7f8e800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e85978640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e80102790 0x7f8e80074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e85978640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e80106f10 0x7f8e80074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e85978640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80111e20 con 0x7f8e80101de0 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e85978640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80111ca0 con 0x7f8e80106f10 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e85978640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80111fa0 con 0x7f8e80102790 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80101de0 0x7f8e800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80101de0 0x7f8e800744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47972/0 (socket says 172.21.15.92:47972) 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 -- 172.21.15.92:0/653069417 learned_addr learned my addr 172.21.15.92:0/653069417 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7e7fc640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e80102790 0x7f8e80074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7f7fe640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e80106f10 0x7f8e80074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 -- 172.21.15.92:0/653069417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e80102790 msgr2=0x7f8e80074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e80102790 0x7f8e80074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 -- 172.21.15.92:0/653069417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e80106f10 msgr2=0x7f8e80074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e80106f10 0x7f8e80074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 -- 172.21.15.92:0/653069417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e801a6820 con 0x7f8e80101de0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7f7fe640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e80106f10 0x7f8e80074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e7effd640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80101de0 0x7f8e800744f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000d770 tx=0x7f8e7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e84976640 1 -- 172.21.15.92:0/653069417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70017070 con 0x7f8e80101de0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.175+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e801a6b10 con 0x7f8e80101de0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.176+0000 7f8e84976640 1 -- 172.21.15.92:0/653069417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e700054d0 con 0x7f8e80101de0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.176+0000 7f8e84976640 1 -- 172.21.15.92:0/653069417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004cf0 con 0x7f8e80101de0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.176+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e801a7050 con 0x7f8e80101de0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.176+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e801021c0 con 0x7f8e80101de0 2024-09-11T22:12:50.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.178+0000 7f8e84976640 1 -- 172.21.15.92:0/653069417 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8e70004e90 con 0x7f8e80101de0 2024-09-11T22:12:50.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.178+0000 7f8e84976640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e680a2c10 0x7f8e680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:50.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.179+0000 7f8e7e7fc640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e680a2c10 0x7f8e680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:50.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.179+0000 7f8e84976640 1 -- 172.21.15.92:0/653069417 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8e700cee80 con 0x7f8e80101de0 2024-09-11T22:12:50.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.179+0000 7f8e7e7fc640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e680a2c10 0x7f8e680a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8e80075f80 tx=0x7f8e7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:50.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.183+0000 7f8e84976640 1 -- 172.21.15.92:0/653069417 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e7009ba40 con 0x7f8e80101de0 2024-09-11T22:12:50.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:49 smithi110 ceph-mon[34830]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:50.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.383+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e8010a150 con 0x7f8e80101de0 2024-09-11T22:12:50.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.384+0000 7f8e84976640 1 -- 172.21.15.92:0/653069417 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11980 (secure 0 0 0) 0x7f8e7009b340 con 0x7f8e80101de0 2024-09-11T22:12:50.385 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:50.386 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"btime":"2024-09-11T22:12:40:147657+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:40.147644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:12:50.386 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-09-11T22:12:50.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e680a2c10 msgr2=0x7f8e680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:50.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e680a2c10 0x7f8e680a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8e80075f80 tx=0x7f8e7400b410 comp rx=0 tx=0).stop 2024-09-11T22:12:50.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80101de0 msgr2=0x7f8e800744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:50.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80101de0 0x7f8e800744f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000d770 tx=0x7f8e7000dc30 comp rx=0 tx=0).stop 2024-09-11T22:12:50.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 shutdown_connections 2024-09-11T22:12:50.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f8e680a2c10 0x7f8e680a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e80106f10 0x7f8e80074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e80102790 0x7f8e80074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 --2- 172.21.15.92:0/653069417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e80101de0 0x7f8e800744f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:50.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 >> 172.21.15.92:0/653069417 conn(0x7f8e800fd8c0 msgr2=0x7f8e800fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:50.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 shutdown_connections 2024-09-11T22:12:50.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:50.386+0000 7f8e85978640 1 -- 172.21.15.92:0/653069417 wait complete. 2024-09-11T22:12:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:50 smithi092 ceph-mon[31173]: pgmap v307: 129 pgs: 129 active+clean; 608 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 KiB/s wr, 24 op/s 2024-09-11T22:12:51.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:50 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/653069417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:50 smithi153 ceph-mon[35045]: pgmap v307: 129 pgs: 129 active+clean; 608 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 KiB/s wr, 24 op/s 2024-09-11T22:12:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:50 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/653069417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:51.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:12:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:50 smithi110 ceph-mon[34830]: pgmap v307: 129 pgs: 129 active+clean; 608 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 KiB/s wr, 24 op/s 2024-09-11T22:12:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:50 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/653069417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:12:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:51.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 -- 172.21.15.92:0/4285006696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88102570 msgr2=0x7fca881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/4285006696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88102570 0x7fca881029b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fca70009900 tx=0x7fca7002a480 comp rx=0 tx=0).stop 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 -- 172.21.15.92:0/4285006696 shutdown_connections 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/4285006696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca88102ef0 0x7fca8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/4285006696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88102570 0x7fca881029b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/4285006696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca88100e20 0x7fca88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 -- 172.21.15.92:0/4285006696 >> 172.21.15.92:0/4285006696 conn(0x7fca880fc7d0 msgr2=0x7fca880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 -- 172.21.15.92:0/4285006696 shutdown_connections 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 -- 172.21.15.92:0/4285006696 wait complete. 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 Processor -- start 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.818+0000 7fca8e6f9640 1 -- start start 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88100e20 0x7fca88199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca88102570 0x7fca881998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca88102ef0 0x7fca881a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca88110ed0 con 0x7fca88100e20 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fca88110d50 con 0x7fca88102ef0 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca88111050 con 0x7fca88102570 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88100e20 0x7fca88199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88100e20 0x7fca88199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47986/0 (socket says 172.21.15.92:47986) 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 -- 172.21.15.92:0/2397737460 learned_addr learned my addr 172.21.15.92:0/2397737460 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca7ffff640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca88102570 0x7fca881998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 -- 172.21.15.92:0/2397737460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca88102570 msgr2=0x7fca881998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca88102570 0x7fca881998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 -- 172.21.15.92:0/2397737460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca88102ef0 msgr2=0x7fca881a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca88102ef0 0x7fca881a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 -- 172.21.15.92:0/2397737460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca881a4940 con 0x7fca88100e20 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca87fff640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88100e20 0x7fca88199380 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b4a0 tx=0x7fca7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca85ffb640 1 -- 172.21.15.92:0/2397737460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78014070 con 0x7fca88100e20 2024-09-11T22:12:51.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca85ffb640 1 -- 172.21.15.92:0/2397737460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca78010ac0 con 0x7fca88100e20 2024-09-11T22:12:51.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca881a4b10 con 0x7fca88100e20 2024-09-11T22:12:51.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca85ffb640 1 -- 172.21.15.92:0/2397737460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78017c30 con 0x7fca88100e20 2024-09-11T22:12:51.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.819+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca881a4fd0 con 0x7fca88100e20 2024-09-11T22:12:51.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.820+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca881021b0 con 0x7fca88100e20 2024-09-11T22:12:51.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.822+0000 7fca85ffb640 1 -- 172.21.15.92:0/2397737460 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca78017070 con 0x7fca88100e20 2024-09-11T22:12:51.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.822+0000 7fca85ffb640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca740a2c10 0x7fca740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:51.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.822+0000 7fca7ffff640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca740a2c10 0x7fca740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:51.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.822+0000 7fca85ffb640 1 -- 172.21.15.92:0/2397737460 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fca780ce7b0 con 0x7fca88100e20 2024-09-11T22:12:51.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.823+0000 7fca7ffff640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca740a2c10 0x7fca740a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca70006010 tx=0x7fca70000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:51.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.826+0000 7fca85ffb640 1 -- 172.21.15.92:0/2397737460 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca7809b480 con 0x7fca88100e20 2024-09-11T22:12:51.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.993+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca88109090 con 0x7fca88100e20 2024-09-11T22:12:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.995+0000 7fca85ffb640 1 -- 172.21.15.92:0/2397737460 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7fca7809ad80 con 0x7fca88100e20 2024-09-11T22:12:51.996 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:51.997 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:12:51.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca740a2c10 msgr2=0x7fca740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:51.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca740a2c10 0x7fca740a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca70006010 tx=0x7fca70000bc0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88100e20 msgr2=0x7fca88199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88100e20 0x7fca88199380 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b4a0 tx=0x7fca7800b960 comp rx=0 tx=0).stop 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 shutdown_connections 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca740a2c10 0x7fca740a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca88102ef0 0x7fca881a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca88102570 0x7fca881998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 --2- 172.21.15.92:0/2397737460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88100e20 0x7fca88199380 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 >> 172.21.15.92:0/2397737460 conn(0x7fca880fc7d0 msgr2=0x7fca880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 shutdown_connections 2024-09-11T22:12:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:51.997+0000 7fca8e6f9640 1 -- 172.21.15.92:0/2397737460 wait complete. 2024-09-11T22:12:52.653 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:12:52.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:12:52.901 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:53.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:52 smithi092 ceph-mon[31173]: pgmap v308: 129 pgs: 129 active+clean; 632 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 20 op/s 2024-09-11T22:12:53.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:52 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2397737460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:53.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:52 smithi153 ceph-mon[35045]: pgmap v308: 129 pgs: 129 active+clean; 632 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 20 op/s 2024-09-11T22:12:53.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:52 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2397737460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:52 smithi110 ceph-mon[34830]: pgmap v308: 129 pgs: 129 active+clean; 632 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 20 op/s 2024-09-11T22:12:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:52 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2397737460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:12:53.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.366+0000 7fa33ddb0640 1 -- 172.21.15.92:0/169401580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338106a80 msgr2=0x7fa338106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:53.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.366+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/169401580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338106a80 0x7fa338106e60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa328009cd0 tx=0x7fa32802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 -- 172.21.15.92:0/169401580 shutdown_connections 2024-09-11T22:12:53.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/169401580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa338101420 0x7fa33810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/169401580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa338100a80 0x7fa338100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/169401580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338106a80 0x7fa338106e60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 -- 172.21.15.92:0/169401580 >> 172.21.15.92:0/169401580 conn(0x7fa3380fc7a0 msgr2=0x7fa3380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 -- 172.21.15.92:0/169401580 shutdown_connections 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 -- 172.21.15.92:0/169401580 wait complete. 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 Processor -- start 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.367+0000 7fa33ddb0640 1 -- start start 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa33ddb0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa338100a80 0x7fa338198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa33ddb0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338101420 0x7fa338198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa33ddb0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa338106a80 0x7fa33819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa33ddb0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa338110dd0 con 0x7fa338101420 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa33ddb0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa338110c50 con 0x7fa338106a80 2024-09-11T22:12:53.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa33ddb0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa338110f50 con 0x7fa338100a80 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338101420 0x7fa338198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338101420 0x7fa338198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48004/0 (socket says 172.21.15.92:48004) 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 -- 172.21.15.92:0/1986501950 learned_addr learned my addr 172.21.15.92:0/1986501950 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa3377fe640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa338100a80 0x7fa338198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa337fff640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa338106a80 0x7fa33819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 -- 172.21.15.92:0/1986501950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa338100a80 msgr2=0x7fa338198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa338100a80 0x7fa338198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 -- 172.21.15.92:0/1986501950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa338106a80 msgr2=0x7fa33819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa338106a80 0x7fa33819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 -- 172.21.15.92:0/1986501950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33819d4b0 con 0x7fa338101420 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa337fff640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa338106a80 0x7fa33819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa336ffd640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338101420 0x7fa338198a40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00b4a0 tx=0x7fa32c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa334ff9640 1 -- 172.21.15.92:0/1986501950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c014070 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa334ff9640 1 -- 172.21.15.92:0/1986501950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa32c010ac0 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33819d7a0 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.368+0000 7fa334ff9640 1 -- 172.21.15.92:0/1986501950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c017c30 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.369+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381a5080 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.370+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa338100f60 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.372+0000 7fa334ff9640 1 -- 172.21.15.92:0/1986501950 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa32c017070 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.372+0000 7fa334ff9640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa3200a2c90 0x7fa3200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.372+0000 7fa3377fe640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa3200a2c90 0x7fa3200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.372+0000 7fa334ff9640 1 -- 172.21.15.92:0/1986501950 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa32c0ce9e0 con 0x7fa338101420 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.373+0000 7fa3377fe640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa3200a2c90 0x7fa3200a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa328006010 tx=0x7fa328005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.375+0000 7fa334ff9640 1 -- 172.21.15.92:0/1986501950 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa32c09b6b0 con 0x7fa338101420 2024-09-11T22:12:53.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.538+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3380fce80 con 0x7fa3200a2c90 2024-09-11T22:12:53.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.547+0000 7fa334ff9640 1 -- 172.21.15.92:0/1986501950 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340704 (secure 0 0 0) 0x7fa3380fce80 con 0x7fa3200a2c90 2024-09-11T22:12:53.552 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T22:12:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa3200a2c90 msgr2=0x7fa3200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa3200a2c90 0x7fa3200a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa328006010 tx=0x7fa328005f70 comp rx=0 tx=0).stop 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338101420 msgr2=0x7fa338198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338101420 0x7fa338198a40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00b4a0 tx=0x7fa32c00b960 comp rx=0 tx=0).stop 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 shutdown_connections 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa3200a2c90 0x7fa3200a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa338106a80 0x7fa33819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa338101420 0x7fa338198a40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 --2- 172.21.15.92:0/1986501950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa338100a80 0x7fa338198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 >> 172.21.15.92:0/1986501950 conn(0x7fa3380fc7a0 msgr2=0x7fa3381049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 shutdown_connections 2024-09-11T22:12:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:53.551+0000 7fa33ddb0640 1 -- 172.21.15.92:0/1986501950 wait complete. 2024-09-11T22:12:54.215 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-11T22:12:51.692911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646908,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":257,"num_read_kb":240,"num_write":416,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382052,"kb_used_data":12004,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559772,"statfs":{"total":1151940427776,"available":1151549206528,"internally_reserved":0,"allocated":12292096,"data_stored":7800009,"data_compressed":326532,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48016,"num_objects":44,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":116,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003510"},"pg_stats":[{"pgid":"2.1f","version":"90'13","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.169054+0000","last_change":"2024-09-11T22:12:00.751331+0000","last_active":"2024-09-11T22:12:47.169054+0000","last_peered":"2024-09-11T22:12:47.169054+0000","last_clean":"2024-09-11T22:12:47.169054+0000","last_became_active":"2024-09-11T22:12:00.751034+0000","last_became_peered":"2024-09-11T22:12:00.751034+0000","last_unstale":"2024-09-11T22:12:47.169054+0000","last_undegraded":"2024-09-11T22:12:47.169054+0000","last_fullsized":"2024-09-11T22:12:47.169054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:14.418754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405924+0000","last_change":"2024-09-11T22:12:02.776052+0000","last_active":"2024-09-11T22:12:13.405924+0000","last_peered":"2024-09-11T22:12:13.405924+0000","last_clean":"2024-09-11T22:12:13.405924+0000","last_became_active":"2024-09-11T22:12:02.775727+0000","last_became_peered":"2024-09-11T22:12:02.775727+0000","last_unstale":"2024-09-11T22:12:13.405924+0000","last_undegraded":"2024-09-11T22:12:13.405924+0000","last_fullsized":"2024-09-11T22:12:13.405924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:50.805895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406006+0000","last_change":"2024-09-11T22:12:09.818054+0000","last_active":"2024-09-11T22:12:13.406006+0000","last_peered":"2024-09-11T22:12:13.406006+0000","last_clean":"2024-09-11T22:12:13.406006+0000","last_became_active":"2024-09-11T22:12:09.817527+0000","last_became_peered":"2024-09-11T22:12:09.817527+0000","last_unstale":"2024-09-11T22:12:13.406006+0000","last_undegraded":"2024-09-11T22:12:13.406006+0000","last_fullsized":"2024-09-11T22:12:13.406006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:26.496214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406116+0000","last_change":"2024-09-11T22:12:00.817451+0000","last_active":"2024-09-11T22:12:13.406116+0000","last_peered":"2024-09-11T22:12:13.406116+0000","last_clean":"2024-09-11T22:12:13.406116+0000","last_became_active":"2024-09-11T22:12:00.817047+0000","last_became_peered":"2024-09-11T22:12:00.817047+0000","last_unstale":"2024-09-11T22:12:13.406116+0000","last_undegraded":"2024-09-11T22:12:13.406116+0000","last_fullsized":"2024-09-11T22:12:13.406116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:48.450520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406368+0000","last_change":"2024-09-11T22:12:02.764638+0000","last_active":"2024-09-11T22:12:13.406368+0000","last_peered":"2024-09-11T22:12:13.406368+0000","last_clean":"2024-09-11T22:12:13.406368+0000","last_became_active":"2024-09-11T22:12:02.764329+0000","last_became_peered":"2024-09-11T22:12:02.764329+0000","last_unstale":"2024-09-11T22:12:13.406368+0000","last_undegraded":"2024-09-11T22:12:13.406368+0000","last_fullsized":"2024-09-11T22:12:13.406368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:27.322976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405814+0000","last_change":"2024-09-11T22:12:09.817478+0000","last_active":"2024-09-11T22:12:13.405814+0000","last_peered":"2024-09-11T22:12:13.405814+0000","last_clean":"2024-09-11T22:12:13.405814+0000","last_became_active":"2024-09-11T22:12:09.817098+0000","last_became_peered":"2024-09-11T22:12:09.817098+0000","last_unstale":"2024-09-11T22:12:13.405814+0000","last_undegraded":"2024-09-11T22:12:13.405814+0000","last_fullsized":"2024-09-11T22:12:13.405814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:50.066572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:40.149294+0000","last_change":"2024-09-11T22:12:00.754280+0000","last_active":"2024-09-11T22:12:40.149294+0000","last_peered":"2024-09-11T22:12:40.149294+0000","last_clean":"2024-09-11T22:12:40.149294+0000","last_became_active":"2024-09-11T22:12:00.753906+0000","last_became_peered":"2024-09-11T22:12:00.753906+0000","last_unstale":"2024-09-11T22:12:40.149294+0000","last_undegraded":"2024-09-11T22:12:40.149294+0000","last_fullsized":"2024-09-11T22:12:40.149294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:01.049033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406363+0000","last_change":"2024-09-11T22:12:02.765037+0000","last_active":"2024-09-11T22:12:13.406363+0000","last_peered":"2024-09-11T22:12:13.406363+0000","last_clean":"2024-09-11T22:12:13.406363+0000","last_became_active":"2024-09-11T22:12:02.764730+0000","last_became_peered":"2024-09-11T22:12:02.764730+0000","last_unstale":"2024-09-11T22:12:13.406363+0000","last_undegraded":"2024-09-11T22:12:13.406363+0000","last_fullsized":"2024-09-11T22:12:13.406363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:13.974389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406695+0000","last_change":"2024-09-11T22:12:09.817801+0000","last_active":"2024-09-11T22:12:13.406695+0000","last_peered":"2024-09-11T22:12:13.406695+0000","last_clean":"2024-09-11T22:12:13.406695+0000","last_became_active":"2024-09-11T22:12:09.817389+0000","last_became_peered":"2024-09-11T22:12:09.817389+0000","last_unstale":"2024-09-11T22:12:13.406695+0000","last_undegraded":"2024-09-11T22:12:13.406695+0000","last_fullsized":"2024-09-11T22:12:13.406695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:55.152113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.155140+0000","last_change":"2024-09-11T22:12:00.752216+0000","last_active":"2024-09-11T22:12:39.155140+0000","last_peered":"2024-09-11T22:12:39.155140+0000","last_clean":"2024-09-11T22:12:39.155140+0000","last_became_active":"2024-09-11T22:12:00.751641+0000","last_became_peered":"2024-09-11T22:12:00.751641+0000","last_unstale":"2024-09-11T22:12:39.155140+0000","last_undegraded":"2024-09-11T22:12:39.155140+0000","last_fullsized":"2024-09-11T22:12:39.155140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814969+0000","last_change":"2024-09-11T22:12:02.776766+0000","last_active":"2024-09-11T22:12:12.814969+0000","last_peered":"2024-09-11T22:12:12.814969+0000","last_clean":"2024-09-11T22:12:12.814969+0000","last_became_active":"2024-09-11T22:12:02.775985+0000","last_became_peered":"2024-09-11T22:12:02.775985+0000","last_unstale":"2024-09-11T22:12:12.814969+0000","last_undegraded":"2024-09-11T22:12:12.814969+0000","last_fullsized":"2024-09-11T22:12:12.814969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:39.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405815+0000","last_change":"2024-09-11T22:12:09.817906+0000","last_active":"2024-09-11T22:12:13.405815+0000","last_peered":"2024-09-11T22:12:13.405815+0000","last_clean":"2024-09-11T22:12:13.405815+0000","last_became_active":"2024-09-11T22:12:09.817404+0000","last_became_peered":"2024-09-11T22:12:09.817404+0000","last_unstale":"2024-09-11T22:12:13.405815+0000","last_undegraded":"2024-09-11T22:12:13.405815+0000","last_fullsized":"2024-09-11T22:12:13.405815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:17:29.957890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814132+0000","last_change":"2024-09-11T22:12:02.770932+0000","last_active":"2024-09-11T22:12:12.814132+0000","last_peered":"2024-09-11T22:12:12.814132+0000","last_clean":"2024-09-11T22:12:12.814132+0000","last_became_active":"2024-09-11T22:12:02.770619+0000","last_became_peered":"2024-09-11T22:12:02.770619+0000","last_unstale":"2024-09-11T22:12:12.814132+0000","last_undegraded":"2024-09-11T22:12:12.814132+0000","last_fullsized":"2024-09-11T22:12:12.814132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:31.746892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:36.129600+0000","last_change":"2024-09-11T22:12:00.754437+0000","last_active":"2024-09-11T22:12:36.129600+0000","last_peered":"2024-09-11T22:12:36.129600+0000","last_clean":"2024-09-11T22:12:36.129600+0000","last_became_active":"2024-09-11T22:12:00.753874+0000","last_became_peered":"2024-09-11T22:12:00.753874+0000","last_unstale":"2024-09-11T22:12:36.129600+0000","last_undegraded":"2024-09-11T22:12:36.129600+0000","last_fullsized":"2024-09-11T22:12:36.129600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:49.305272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406481+0000","last_change":"2024-09-11T22:12:10.199589+0000","last_active":"2024-09-11T22:12:13.406481+0000","last_peered":"2024-09-11T22:12:13.406481+0000","last_clean":"2024-09-11T22:12:13.406481+0000","last_became_active":"2024-09-11T22:12:10.199185+0000","last_became_peered":"2024-09-11T22:12:10.199185+0000","last_unstale":"2024-09-11T22:12:13.406481+0000","last_undegraded":"2024-09-11T22:12:13.406481+0000","last_fullsized":"2024-09-11T22:12:13.406481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:51.081016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405566+0000","last_change":"2024-09-11T22:12:02.763355+0000","last_active":"2024-09-11T22:12:13.405566+0000","last_peered":"2024-09-11T22:12:13.405566+0000","last_clean":"2024-09-11T22:12:13.405566+0000","last_became_active":"2024-09-11T22:12:02.763060+0000","last_became_peered":"2024-09-11T22:12:02.763060+0000","last_unstale":"2024-09-11T22:12:13.405566+0000","last_undegraded":"2024-09-11T22:12:13.405566+0000","last_fullsized":"2024-09-11T22:12:13.405566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:22.776503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:38.143624+0000","last_change":"2024-09-11T22:12:00.754158+0000","last_active":"2024-09-11T22:12:38.143624+0000","last_peered":"2024-09-11T22:12:38.143624+0000","last_clean":"2024-09-11T22:12:38.143624+0000","last_became_active":"2024-09-11T22:12:00.753706+0000","last_became_peered":"2024-09-11T22:12:00.753706+0000","last_unstale":"2024-09-11T22:12:38.143624+0000","last_undegraded":"2024-09-11T22:12:38.143624+0000","last_fullsized":"2024-09-11T22:12:38.143624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:40.470704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406214+0000","last_change":"2024-09-11T22:12:09.810374+0000","last_active":"2024-09-11T22:12:13.406214+0000","last_peered":"2024-09-11T22:12:13.406214+0000","last_clean":"2024-09-11T22:12:13.406214+0000","last_became_active":"2024-09-11T22:12:09.809915+0000","last_became_peered":"2024-09-11T22:12:09.809915+0000","last_unstale":"2024-09-11T22:12:13.406214+0000","last_undegraded":"2024-09-11T22:12:13.406214+0000","last_fullsized":"2024-09-11T22:12:13.406214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:25.554920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699035+0000","last_change":"2024-09-11T22:12:02.769350+0000","last_active":"2024-09-11T22:12:13.699035+0000","last_peered":"2024-09-11T22:12:13.699035+0000","last_clean":"2024-09-11T22:12:13.699035+0000","last_became_active":"2024-09-11T22:12:02.768896+0000","last_became_peered":"2024-09-11T22:12:02.768896+0000","last_unstale":"2024-09-11T22:12:13.699035+0000","last_undegraded":"2024-09-11T22:12:13.699035+0000","last_fullsized":"2024-09-11T22:12:13.699035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:12.490938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.150797+0000","last_change":"2024-09-11T22:12:00.743281+0000","last_active":"2024-09-11T22:12:39.150797+0000","last_peered":"2024-09-11T22:12:39.150797+0000","last_clean":"2024-09-11T22:12:39.150797+0000","last_became_active":"2024-09-11T22:12:00.742977+0000","last_became_peered":"2024-09-11T22:12:00.742977+0000","last_unstale":"2024-09-11T22:12:39.150797+0000","last_undegraded":"2024-09-11T22:12:39.150797+0000","last_fullsized":"2024-09-11T22:12:39.150797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:28.890328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206889+0000","last_change":"2024-09-11T22:12:09.815471+0000","last_active":"2024-09-11T22:12:13.206889+0000","last_peered":"2024-09-11T22:12:13.206889+0000","last_clean":"2024-09-11T22:12:13.206889+0000","last_became_active":"2024-09-11T22:12:09.815098+0000","last_became_peered":"2024-09-11T22:12:09.815098+0000","last_unstale":"2024-09-11T22:12:13.206889+0000","last_undegraded":"2024-09-11T22:12:13.206889+0000","last_fullsized":"2024-09-11T22:12:13.206889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:36.375848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405512+0000","last_change":"2024-09-11T22:12:02.763597+0000","last_active":"2024-09-11T22:12:13.405512+0000","last_peered":"2024-09-11T22:12:13.405512+0000","last_clean":"2024-09-11T22:12:13.405512+0000","last_became_active":"2024-09-11T22:12:02.763242+0000","last_became_peered":"2024-09-11T22:12:02.763242+0000","last_unstale":"2024-09-11T22:12:13.405512+0000","last_undegraded":"2024-09-11T22:12:13.405512+0000","last_fullsized":"2024-09-11T22:12:13.405512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:24.968799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:36.128741+0000","last_change":"2024-09-11T22:12:00.817604+0000","last_active":"2024-09-11T22:12:36.128741+0000","last_peered":"2024-09-11T22:12:36.128741+0000","last_clean":"2024-09-11T22:12:36.128741+0000","last_became_active":"2024-09-11T22:12:00.817377+0000","last_became_peered":"2024-09-11T22:12:00.817377+0000","last_unstale":"2024-09-11T22:12:36.128741+0000","last_undegraded":"2024-09-11T22:12:36.128741+0000","last_fullsized":"2024-09-11T22:12:36.128741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:27.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406696+0000","last_change":"2024-09-11T22:12:10.201074+0000","last_active":"2024-09-11T22:12:13.406696+0000","last_peered":"2024-09-11T22:12:13.406696+0000","last_clean":"2024-09-11T22:12:13.406696+0000","last_became_active":"2024-09-11T22:12:10.200664+0000","last_became_peered":"2024-09-11T22:12:10.200664+0000","last_unstale":"2024-09-11T22:12:13.406696+0000","last_undegraded":"2024-09-11T22:12:13.406696+0000","last_fullsized":"2024-09-11T22:12:13.406696+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:44.784751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406118+0000","last_change":"2024-09-11T22:12:02.773380+0000","last_active":"2024-09-11T22:12:13.406118+0000","last_peered":"2024-09-11T22:12:13.406118+0000","last_clean":"2024-09-11T22:12:13.406118+0000","last_became_active":"2024-09-11T22:12:02.773076+0000","last_became_peered":"2024-09-11T22:12:02.773076+0000","last_unstale":"2024-09-11T22:12:13.406118+0000","last_undegraded":"2024-09-11T22:12:13.406118+0000","last_fullsized":"2024-09-11T22:12:13.406118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:35.475581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.152455+0000","last_change":"2024-09-11T22:12:00.746438+0000","last_active":"2024-09-11T22:12:39.152455+0000","last_peered":"2024-09-11T22:12:39.152455+0000","last_clean":"2024-09-11T22:12:39.152455+0000","last_became_active":"2024-09-11T22:12:00.746034+0000","last_became_peered":"2024-09-11T22:12:00.746034+0000","last_unstale":"2024-09-11T22:12:39.152455+0000","last_undegraded":"2024-09-11T22:12:39.152455+0000","last_fullsized":"2024-09-11T22:12:39.152455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:49.156748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406088+0000","last_change":"2024-09-11T22:12:09.812553+0000","last_active":"2024-09-11T22:12:13.406088+0000","last_peered":"2024-09-11T22:12:13.406088+0000","last_clean":"2024-09-11T22:12:13.406088+0000","last_became_active":"2024-09-11T22:12:09.812092+0000","last_became_peered":"2024-09-11T22:12:09.812092+0000","last_unstale":"2024-09-11T22:12:13.406088+0000","last_undegraded":"2024-09-11T22:12:13.406088+0000","last_fullsized":"2024-09-11T22:12:13.406088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:13.947799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406006+0000","last_change":"2024-09-11T22:12:02.763399+0000","last_active":"2024-09-11T22:12:13.406006+0000","last_peered":"2024-09-11T22:12:13.406006+0000","last_clean":"2024-09-11T22:12:13.406006+0000","last_became_active":"2024-09-11T22:12:02.763082+0000","last_became_peered":"2024-09-11T22:12:02.763082+0000","last_unstale":"2024-09-11T22:12:13.406006+0000","last_undegraded":"2024-09-11T22:12:13.406006+0000","last_fullsized":"2024-09-11T22:12:13.406006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:52.267960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.776645+0000","last_change":"2024-09-11T22:12:00.751318+0000","last_active":"2024-09-11T22:12:47.776645+0000","last_peered":"2024-09-11T22:12:47.776645+0000","last_clean":"2024-09-11T22:12:47.776645+0000","last_became_active":"2024-09-11T22:12:00.751022+0000","last_became_peered":"2024-09-11T22:12:00.751022+0000","last_unstale":"2024-09-11T22:12:47.776645+0000","last_undegraded":"2024-09-11T22:12:47.776645+0000","last_fullsized":"2024-09-11T22:12:47.776645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:23:26.163502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405558+0000","last_change":"2024-09-11T22:12:10.200279+0000","last_active":"2024-09-11T22:12:13.405558+0000","last_peered":"2024-09-11T22:12:13.405558+0000","last_clean":"2024-09-11T22:12:13.405558+0000","last_became_active":"2024-09-11T22:12:10.199920+0000","last_became_peered":"2024-09-11T22:12:10.199920+0000","last_unstale":"2024-09-11T22:12:13.405558+0000","last_undegraded":"2024-09-11T22:12:13.405558+0000","last_fullsized":"2024-09-11T22:12:13.405558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:59.012735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814148+0000","last_change":"2024-09-11T22:12:02.755565+0000","last_active":"2024-09-11T22:12:12.814148+0000","last_peered":"2024-09-11T22:12:12.814148+0000","last_clean":"2024-09-11T22:12:12.814148+0000","last_became_active":"2024-09-11T22:12:02.755236+0000","last_became_peered":"2024-09-11T22:12:02.755236+0000","last_unstale":"2024-09-11T22:12:12.814148+0000","last_undegraded":"2024-09-11T22:12:12.814148+0000","last_fullsized":"2024-09-11T22:12:12.814148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.588976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.154310+0000","last_change":"2024-09-11T22:12:00.818634+0000","last_active":"2024-09-11T22:12:39.154310+0000","last_peered":"2024-09-11T22:12:39.154310+0000","last_clean":"2024-09-11T22:12:39.154310+0000","last_became_active":"2024-09-11T22:12:00.817917+0000","last_became_peered":"2024-09-11T22:12:00.817917+0000","last_unstale":"2024-09-11T22:12:39.154310+0000","last_undegraded":"2024-09-11T22:12:39.154310+0000","last_fullsized":"2024-09-11T22:12:39.154310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:42.382962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699203+0000","last_change":"2024-09-11T22:12:09.814770+0000","last_active":"2024-09-11T22:12:13.699203+0000","last_peered":"2024-09-11T22:12:13.699203+0000","last_clean":"2024-09-11T22:12:13.699203+0000","last_became_active":"2024-09-11T22:12:09.814322+0000","last_became_peered":"2024-09-11T22:12:09.814322+0000","last_unstale":"2024-09-11T22:12:13.699203+0000","last_undegraded":"2024-09-11T22:12:13.699203+0000","last_fullsized":"2024-09-11T22:12:13.699203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.666161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.407233+0000","last_change":"2024-09-11T22:12:02.762939+0000","last_active":"2024-09-11T22:12:13.407233+0000","last_peered":"2024-09-11T22:12:13.407233+0000","last_clean":"2024-09-11T22:12:13.407233+0000","last_became_active":"2024-09-11T22:12:02.762579+0000","last_became_peered":"2024-09-11T22:12:02.762579+0000","last_unstale":"2024-09-11T22:12:13.407233+0000","last_undegraded":"2024-09-11T22:12:13.407233+0000","last_fullsized":"2024-09-11T22:12:13.407233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:35.124612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206779+0000","last_change":"2024-09-11T22:12:00.818094+0000","last_active":"2024-09-11T22:12:13.206779+0000","last_peered":"2024-09-11T22:12:13.206779+0000","last_clean":"2024-09-11T22:12:13.206779+0000","last_became_active":"2024-09-11T22:12:00.817723+0000","last_became_peered":"2024-09-11T22:12:00.817723+0000","last_unstale":"2024-09-11T22:12:13.206779+0000","last_undegraded":"2024-09-11T22:12:13.206779+0000","last_fullsized":"2024-09-11T22:12:13.206779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:32.618689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405603+0000","last_change":"2024-09-11T22:12:10.199778+0000","last_active":"2024-09-11T22:12:13.405603+0000","last_peered":"2024-09-11T22:12:13.405603+0000","last_clean":"2024-09-11T22:12:13.405603+0000","last_became_active":"2024-09-11T22:12:10.199280+0000","last_became_peered":"2024-09-11T22:12:10.199280+0000","last_unstale":"2024-09-11T22:12:13.405603+0000","last_undegraded":"2024-09-11T22:12:13.405603+0000","last_fullsized":"2024-09-11T22:12:13.405603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:08.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406572+0000","last_change":"2024-09-11T22:12:02.769477+0000","last_active":"2024-09-11T22:12:13.406572+0000","last_peered":"2024-09-11T22:12:13.406572+0000","last_clean":"2024-09-11T22:12:13.406572+0000","last_became_active":"2024-09-11T22:12:02.768847+0000","last_became_peered":"2024-09-11T22:12:02.768847+0000","last_unstale":"2024-09-11T22:12:13.406572+0000","last_undegraded":"2024-09-11T22:12:13.406572+0000","last_fullsized":"2024-09-11T22:12:13.406572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:57.612115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.169516+0000","last_change":"2024-09-11T22:12:00.744559+0000","last_active":"2024-09-11T22:12:47.169516+0000","last_peered":"2024-09-11T22:12:47.169516+0000","last_clean":"2024-09-11T22:12:47.169516+0000","last_became_active":"2024-09-11T22:12:00.744266+0000","last_became_peered":"2024-09-11T22:12:00.744266+0000","last_unstale":"2024-09-11T22:12:47.169516+0000","last_undegraded":"2024-09-11T22:12:47.169516+0000","last_fullsized":"2024-09-11T22:12:47.169516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:12.663275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814107+0000","last_change":"2024-09-11T22:12:09.819627+0000","last_active":"2024-09-11T22:12:12.814107+0000","last_peered":"2024-09-11T22:12:12.814107+0000","last_clean":"2024-09-11T22:12:12.814107+0000","last_became_active":"2024-09-11T22:12:09.819241+0000","last_became_peered":"2024-09-11T22:12:09.819241+0000","last_unstale":"2024-09-11T22:12:12.814107+0000","last_undegraded":"2024-09-11T22:12:12.814107+0000","last_fullsized":"2024-09-11T22:12:12.814107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:32.932384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814449+0000","last_change":"2024-09-11T22:12:02.776860+0000","last_active":"2024-09-11T22:12:12.814449+0000","last_peered":"2024-09-11T22:12:12.814449+0000","last_clean":"2024-09-11T22:12:12.814449+0000","last_became_active":"2024-09-11T22:12:02.776421+0000","last_became_peered":"2024-09-11T22:12:02.776421+0000","last_unstale":"2024-09-11T22:12:12.814449+0000","last_undegraded":"2024-09-11T22:12:12.814449+0000","last_fullsized":"2024-09-11T22:12:12.814449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:23:43.022309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.169340+0000","last_change":"2024-09-11T22:12:00.817499+0000","last_active":"2024-09-11T22:12:47.169340+0000","last_peered":"2024-09-11T22:12:47.169340+0000","last_clean":"2024-09-11T22:12:47.169340+0000","last_became_active":"2024-09-11T22:12:00.817152+0000","last_became_peered":"2024-09-11T22:12:00.817152+0000","last_unstale":"2024-09-11T22:12:47.169340+0000","last_undegraded":"2024-09-11T22:12:47.169340+0000","last_fullsized":"2024-09-11T22:12:47.169340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:19.716267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813879+0000","last_change":"2024-09-11T22:12:10.200318+0000","last_active":"2024-09-11T22:12:12.813879+0000","last_peered":"2024-09-11T22:12:12.813879+0000","last_clean":"2024-09-11T22:12:12.813879+0000","last_became_active":"2024-09-11T22:12:10.199904+0000","last_became_peered":"2024-09-11T22:12:10.199904+0000","last_unstale":"2024-09-11T22:12:12.813879+0000","last_undegraded":"2024-09-11T22:12:12.813879+0000","last_fullsized":"2024-09-11T22:12:12.813879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:47.061489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814023+0000","last_change":"2024-09-11T22:12:02.758872+0000","last_active":"2024-09-11T22:12:12.814023+0000","last_peered":"2024-09-11T22:12:12.814023+0000","last_clean":"2024-09-11T22:12:12.814023+0000","last_became_active":"2024-09-11T22:12:02.758578+0000","last_became_peered":"2024-09-11T22:12:02.758578+0000","last_unstale":"2024-09-11T22:12:12.814023+0000","last_undegraded":"2024-09-11T22:12:12.814023+0000","last_fullsized":"2024-09-11T22:12:12.814023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:51.875760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:37.134811+0000","last_change":"2024-09-11T22:12:00.749202+0000","last_active":"2024-09-11T22:12:37.134811+0000","last_peered":"2024-09-11T22:12:37.134811+0000","last_clean":"2024-09-11T22:12:37.134811+0000","last_became_active":"2024-09-11T22:12:00.748886+0000","last_became_peered":"2024-09-11T22:12:00.748886+0000","last_unstale":"2024-09-11T22:12:37.134811+0000","last_undegraded":"2024-09-11T22:12:37.134811+0000","last_fullsized":"2024-09-11T22:12:37.134811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:44.581466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.815165+0000","last_change":"2024-09-11T22:12:10.201018+0000","last_active":"2024-09-11T22:12:12.815165+0000","last_peered":"2024-09-11T22:12:12.815165+0000","last_clean":"2024-09-11T22:12:12.815165+0000","last_became_active":"2024-09-11T22:12:10.200658+0000","last_became_peered":"2024-09-11T22:12:10.200658+0000","last_unstale":"2024-09-11T22:12:12.815165+0000","last_undegraded":"2024-09-11T22:12:12.815165+0000","last_fullsized":"2024-09-11T22:12:12.815165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.919652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405760+0000","last_change":"2024-09-11T22:12:02.771546+0000","last_active":"2024-09-11T22:12:13.405760+0000","last_peered":"2024-09-11T22:12:13.405760+0000","last_clean":"2024-09-11T22:12:13.405760+0000","last_became_active":"2024-09-11T22:12:02.771296+0000","last_became_peered":"2024-09-11T22:12:02.771296+0000","last_unstale":"2024-09-11T22:12:13.405760+0000","last_undegraded":"2024-09-11T22:12:13.405760+0000","last_fullsized":"2024-09-11T22:12:13.405760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:05.144166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:40.156378+0000","last_change":"2024-09-11T22:12:00.744431+0000","last_active":"2024-09-11T22:12:40.156378+0000","last_peered":"2024-09-11T22:12:40.156378+0000","last_clean":"2024-09-11T22:12:40.156378+0000","last_became_active":"2024-09-11T22:12:00.744121+0000","last_became_peered":"2024-09-11T22:12:00.744121+0000","last_unstale":"2024-09-11T22:12:40.156378+0000","last_undegraded":"2024-09-11T22:12:40.156378+0000","last_fullsized":"2024-09-11T22:12:40.156378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:16.832914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814329+0000","last_change":"2024-09-11T22:10:51.241028+0000","last_active":"2024-09-11T22:12:12.814329+0000","last_peered":"2024-09-11T22:12:12.814329+0000","last_clean":"2024-09-11T22:12:12.814329+0000","last_became_active":"2024-09-11T22:10:51.240533+0000","last_became_peered":"2024-09-11T22:10:51.240533+0000","last_unstale":"2024-09-11T22:12:12.814329+0000","last_undegraded":"2024-09-11T22:12:12.814329+0000","last_fullsized":"2024-09-11T22:12:12.814329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_clean_scrub_stamp":"2024-09-11T22:07:30.402427+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:59.522909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.465088+0000","last_change":"2024-09-11T22:12:09.815386+0000","last_active":"2024-09-11T22:12:13.465088+0000","last_peered":"2024-09-11T22:12:13.465088+0000","last_clean":"2024-09-11T22:12:13.465088+0000","last_became_active":"2024-09-11T22:12:09.815004+0000","last_became_peered":"2024-09-11T22:12:09.815004+0000","last_unstale":"2024-09-11T22:12:13.465088+0000","last_undegraded":"2024-09-11T22:12:13.465088+0000","last_fullsized":"2024-09-11T22:12:13.465088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:22.247309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814644+0000","last_change":"2024-09-11T22:12:02.774139+0000","last_active":"2024-09-11T22:12:12.814644+0000","last_peered":"2024-09-11T22:12:12.814644+0000","last_clean":"2024-09-11T22:12:12.814644+0000","last_became_active":"2024-09-11T22:12:02.773785+0000","last_became_peered":"2024-09-11T22:12:02.773785+0000","last_unstale":"2024-09-11T22:12:12.814644+0000","last_undegraded":"2024-09-11T22:12:12.814644+0000","last_fullsized":"2024-09-11T22:12:12.814644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:53.160278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.170175+0000","last_change":"2024-09-11T22:12:00.818813+0000","last_active":"2024-09-11T22:12:47.170175+0000","last_peered":"2024-09-11T22:12:47.170175+0000","last_clean":"2024-09-11T22:12:47.170175+0000","last_became_active":"2024-09-11T22:12:00.817574+0000","last_became_peered":"2024-09-11T22:12:00.817574+0000","last_unstale":"2024-09-11T22:12:47.170175+0000","last_undegraded":"2024-09-11T22:12:47.170175+0000","last_fullsized":"2024-09-11T22:12:47.170175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:52.429428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406089+0000","last_change":"2024-09-11T22:12:09.819009+0000","last_active":"2024-09-11T22:12:13.406089+0000","last_peered":"2024-09-11T22:12:13.406089+0000","last_clean":"2024-09-11T22:12:13.406089+0000","last_became_active":"2024-09-11T22:12:09.818513+0000","last_became_peered":"2024-09-11T22:12:09.818513+0000","last_unstale":"2024-09-11T22:12:13.406089+0000","last_undegraded":"2024-09-11T22:12:13.406089+0000","last_fullsized":"2024-09-11T22:12:13.406089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:34.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814408+0000","last_change":"2024-09-11T22:12:02.776027+0000","last_active":"2024-09-11T22:12:12.814408+0000","last_peered":"2024-09-11T22:12:12.814408+0000","last_clean":"2024-09-11T22:12:12.814408+0000","last_became_active":"2024-09-11T22:12:02.775485+0000","last_became_peered":"2024-09-11T22:12:02.775485+0000","last_unstale":"2024-09-11T22:12:12.814408+0000","last_undegraded":"2024-09-11T22:12:12.814408+0000","last_fullsized":"2024-09-11T22:12:12.814408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:02.801205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.153793+0000","last_change":"2024-09-11T22:12:00.747291+0000","last_active":"2024-09-11T22:12:39.153793+0000","last_peered":"2024-09-11T22:12:39.153793+0000","last_clean":"2024-09-11T22:12:39.153793+0000","last_became_active":"2024-09-11T22:12:00.746910+0000","last_became_peered":"2024-09-11T22:12:00.746910+0000","last_unstale":"2024-09-11T22:12:39.153793+0000","last_undegraded":"2024-09-11T22:12:39.153793+0000","last_fullsized":"2024-09-11T22:12:39.153793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:23.993924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405817+0000","last_change":"2024-09-11T22:12:09.813999+0000","last_active":"2024-09-11T22:12:13.405817+0000","last_peered":"2024-09-11T22:12:13.405817+0000","last_clean":"2024-09-11T22:12:13.405817+0000","last_became_active":"2024-09-11T22:12:09.813626+0000","last_became_peered":"2024-09-11T22:12:09.813626+0000","last_unstale":"2024-09-11T22:12:13.405817+0000","last_undegraded":"2024-09-11T22:12:13.405817+0000","last_fullsized":"2024-09-11T22:12:13.405817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:12.069515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814677+0000","last_change":"2024-09-11T22:12:02.766451+0000","last_active":"2024-09-11T22:12:12.814677+0000","last_peered":"2024-09-11T22:12:12.814677+0000","last_clean":"2024-09-11T22:12:12.814677+0000","last_became_active":"2024-09-11T22:12:02.765909+0000","last_became_peered":"2024-09-11T22:12:02.765909+0000","last_unstale":"2024-09-11T22:12:12.814677+0000","last_undegraded":"2024-09-11T22:12:12.814677+0000","last_fullsized":"2024-09-11T22:12:12.814677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:34.099196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.151594+0000","last_change":"2024-09-11T22:12:00.748989+0000","last_active":"2024-09-11T22:12:39.151594+0000","last_peered":"2024-09-11T22:12:39.151594+0000","last_clean":"2024-09-11T22:12:39.151594+0000","last_became_active":"2024-09-11T22:12:00.748653+0000","last_became_peered":"2024-09-11T22:12:00.748653+0000","last_unstale":"2024-09-11T22:12:39.151594+0000","last_undegraded":"2024-09-11T22:12:39.151594+0000","last_fullsized":"2024-09-11T22:12:39.151594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:53.234719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814331+0000","last_change":"2024-09-11T22:12:09.818187+0000","last_active":"2024-09-11T22:12:12.814331+0000","last_peered":"2024-09-11T22:12:12.814331+0000","last_clean":"2024-09-11T22:12:12.814331+0000","last_became_active":"2024-09-11T22:12:09.817355+0000","last_became_peered":"2024-09-11T22:12:09.817355+0000","last_unstale":"2024-09-11T22:12:12.814331+0000","last_undegraded":"2024-09-11T22:12:12.814331+0000","last_fullsized":"2024-09-11T22:12:12.814331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:57.355716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699334+0000","last_change":"2024-09-11T22:12:02.774309+0000","last_active":"2024-09-11T22:12:13.699334+0000","last_peered":"2024-09-11T22:12:13.699334+0000","last_clean":"2024-09-11T22:12:13.699334+0000","last_became_active":"2024-09-11T22:12:02.774004+0000","last_became_peered":"2024-09-11T22:12:02.774004+0000","last_unstale":"2024-09-11T22:12:13.699334+0000","last_undegraded":"2024-09-11T22:12:13.699334+0000","last_fullsized":"2024-09-11T22:12:13.699334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:03.107880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.129039+0000","last_change":"2024-09-11T22:12:00.749564+0000","last_active":"2024-09-11T22:12:39.129039+0000","last_peered":"2024-09-11T22:12:39.129039+0000","last_clean":"2024-09-11T22:12:39.129039+0000","last_became_active":"2024-09-11T22:12:00.749221+0000","last_became_peered":"2024-09-11T22:12:00.749221+0000","last_unstale":"2024-09-11T22:12:39.129039+0000","last_undegraded":"2024-09-11T22:12:39.129039+0000","last_fullsized":"2024-09-11T22:12:39.129039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:42:10.875683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814553+0000","last_change":"2024-09-11T22:12:09.821945+0000","last_active":"2024-09-11T22:12:12.814553+0000","last_peered":"2024-09-11T22:12:12.814553+0000","last_clean":"2024-09-11T22:12:12.814553+0000","last_became_active":"2024-09-11T22:12:09.821537+0000","last_became_peered":"2024-09-11T22:12:09.821537+0000","last_unstale":"2024-09-11T22:12:12.814553+0000","last_undegraded":"2024-09-11T22:12:12.814553+0000","last_fullsized":"2024-09-11T22:12:12.814553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:18.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814459+0000","last_change":"2024-09-11T22:12:02.768511+0000","last_active":"2024-09-11T22:12:12.814459+0000","last_peered":"2024-09-11T22:12:12.814459+0000","last_clean":"2024-09-11T22:12:12.814459+0000","last_became_active":"2024-09-11T22:12:02.768060+0000","last_became_peered":"2024-09-11T22:12:02.768060+0000","last_unstale":"2024-09-11T22:12:12.814459+0000","last_undegraded":"2024-09-11T22:12:12.814459+0000","last_fullsized":"2024-09-11T22:12:12.814459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:26.417066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.153490+0000","last_change":"2024-09-11T22:12:00.817941+0000","last_active":"2024-09-11T22:12:39.153490+0000","last_peered":"2024-09-11T22:12:39.153490+0000","last_clean":"2024-09-11T22:12:39.153490+0000","last_became_active":"2024-09-11T22:12:00.817606+0000","last_became_peered":"2024-09-11T22:12:00.817606+0000","last_unstale":"2024-09-11T22:12:39.153490+0000","last_undegraded":"2024-09-11T22:12:39.153490+0000","last_fullsized":"2024-09-11T22:12:39.153490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:44.212961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814031+0000","last_change":"2024-09-11T22:12:09.812516+0000","last_active":"2024-09-11T22:12:12.814031+0000","last_peered":"2024-09-11T22:12:12.814031+0000","last_clean":"2024-09-11T22:12:12.814031+0000","last_became_active":"2024-09-11T22:12:09.812217+0000","last_became_peered":"2024-09-11T22:12:09.812217+0000","last_unstale":"2024-09-11T22:12:12.814031+0000","last_undegraded":"2024-09-11T22:12:12.814031+0000","last_fullsized":"2024-09-11T22:12:12.814031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:36.568803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813978+0000","last_change":"2024-09-11T22:12:02.775797+0000","last_active":"2024-09-11T22:12:12.813978+0000","last_peered":"2024-09-11T22:12:12.813978+0000","last_clean":"2024-09-11T22:12:12.813978+0000","last_became_active":"2024-09-11T22:12:02.775474+0000","last_became_peered":"2024-09-11T22:12:02.775474+0000","last_unstale":"2024-09-11T22:12:12.813978+0000","last_undegraded":"2024-09-11T22:12:12.813978+0000","last_fullsized":"2024-09-11T22:12:12.813978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:25.446633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:46.168619+0000","last_change":"2024-09-11T22:12:00.744237+0000","last_active":"2024-09-11T22:12:46.168619+0000","last_peered":"2024-09-11T22:12:46.168619+0000","last_clean":"2024-09-11T22:12:46.168619+0000","last_became_active":"2024-09-11T22:12:00.743783+0000","last_became_peered":"2024-09-11T22:12:00.743783+0000","last_unstale":"2024-09-11T22:12:46.168619+0000","last_undegraded":"2024-09-11T22:12:46.168619+0000","last_fullsized":"2024-09-11T22:12:46.168619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:53.046685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206953+0000","last_change":"2024-09-11T22:12:09.815279+0000","last_active":"2024-09-11T22:12:13.206953+0000","last_peered":"2024-09-11T22:12:13.206953+0000","last_clean":"2024-09-11T22:12:13.206953+0000","last_became_active":"2024-09-11T22:12:09.814447+0000","last_became_peered":"2024-09-11T22:12:09.814447+0000","last_unstale":"2024-09-11T22:12:13.206953+0000","last_undegraded":"2024-09-11T22:12:13.206953+0000","last_fullsized":"2024-09-11T22:12:13.206953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.289292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814217+0000","last_change":"2024-09-11T22:12:02.769205+0000","last_active":"2024-09-11T22:12:12.814217+0000","last_peered":"2024-09-11T22:12:12.814217+0000","last_clean":"2024-09-11T22:12:12.814217+0000","last_became_active":"2024-09-11T22:12:02.768890+0000","last_became_peered":"2024-09-11T22:12:02.768890+0000","last_unstale":"2024-09-11T22:12:12.814217+0000","last_undegraded":"2024-09-11T22:12:12.814217+0000","last_fullsized":"2024-09-11T22:12:12.814217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:43.705862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.151899+0000","last_change":"2024-09-11T22:12:00.749008+0000","last_active":"2024-09-11T22:12:39.151899+0000","last_peered":"2024-09-11T22:12:39.151899+0000","last_clean":"2024-09-11T22:12:39.151899+0000","last_became_active":"2024-09-11T22:12:00.748642+0000","last_became_peered":"2024-09-11T22:12:00.748642+0000","last_unstale":"2024-09-11T22:12:39.151899+0000","last_undegraded":"2024-09-11T22:12:39.151899+0000","last_fullsized":"2024-09-11T22:12:39.151899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:34.763101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699172+0000","last_change":"2024-09-11T22:12:09.816925+0000","last_active":"2024-09-11T22:12:13.699172+0000","last_peered":"2024-09-11T22:12:13.699172+0000","last_clean":"2024-09-11T22:12:13.699172+0000","last_became_active":"2024-09-11T22:12:09.816434+0000","last_became_peered":"2024-09-11T22:12:09.816434+0000","last_unstale":"2024-09-11T22:12:13.699172+0000","last_undegraded":"2024-09-11T22:12:13.699172+0000","last_fullsized":"2024-09-11T22:12:13.699172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813718+0000","last_change":"2024-09-11T22:12:02.761721+0000","last_active":"2024-09-11T22:12:12.813718+0000","last_peered":"2024-09-11T22:12:12.813718+0000","last_clean":"2024-09-11T22:12:12.813718+0000","last_became_active":"2024-09-11T22:12:02.761448+0000","last_became_peered":"2024-09-11T22:12:02.761448+0000","last_unstale":"2024-09-11T22:12:12.813718+0000","last_undegraded":"2024-09-11T22:12:12.813718+0000","last_fullsized":"2024-09-11T22:12:12.813718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:51.401729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:40.156667+0000","last_change":"2024-09-11T22:12:00.751110+0000","last_active":"2024-09-11T22:12:40.156667+0000","last_peered":"2024-09-11T22:12:40.156667+0000","last_clean":"2024-09-11T22:12:40.156667+0000","last_became_active":"2024-09-11T22:12:00.749431+0000","last_became_peered":"2024-09-11T22:12:00.749431+0000","last_unstale":"2024-09-11T22:12:40.156667+0000","last_undegraded":"2024-09-11T22:12:40.156667+0000","last_fullsized":"2024-09-11T22:12:40.156667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:27.774865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.815091+0000","last_change":"2024-09-11T22:12:09.820852+0000","last_active":"2024-09-11T22:12:12.815091+0000","last_peered":"2024-09-11T22:12:12.815091+0000","last_clean":"2024-09-11T22:12:12.815091+0000","last_became_active":"2024-09-11T22:12:09.820263+0000","last_became_peered":"2024-09-11T22:12:09.820263+0000","last_unstale":"2024-09-11T22:12:12.815091+0000","last_undegraded":"2024-09-11T22:12:12.815091+0000","last_fullsized":"2024-09-11T22:12:12.815091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:09.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406473+0000","last_change":"2024-09-11T22:12:02.770819+0000","last_active":"2024-09-11T22:12:13.406473+0000","last_peered":"2024-09-11T22:12:13.406473+0000","last_clean":"2024-09-11T22:12:13.406473+0000","last_became_active":"2024-09-11T22:12:02.770502+0000","last_became_peered":"2024-09-11T22:12:02.770502+0000","last_unstale":"2024-09-11T22:12:13.406473+0000","last_undegraded":"2024-09-11T22:12:13.406473+0000","last_fullsized":"2024-09-11T22:12:13.406473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:10.345880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'28","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:49.174220+0000","last_change":"2024-09-11T22:12:00.747739+0000","last_active":"2024-09-11T22:12:49.174220+0000","last_peered":"2024-09-11T22:12:49.174220+0000","last_clean":"2024-09-11T22:12:49.174220+0000","last_became_active":"2024-09-11T22:12:00.747372+0000","last_became_peered":"2024-09-11T22:12:00.747372+0000","last_unstale":"2024-09-11T22:12:49.174220+0000","last_undegraded":"2024-09-11T22:12:49.174220+0000","last_fullsized":"2024-09-11T22:12:49.174220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:12.943134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":30,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206504+0000","last_change":"2024-09-11T22:12:09.819643+0000","last_active":"2024-09-11T22:12:13.206504+0000","last_peered":"2024-09-11T22:12:13.206504+0000","last_clean":"2024-09-11T22:12:13.206504+0000","last_became_active":"2024-09-11T22:12:09.819144+0000","last_became_peered":"2024-09-11T22:12:09.819144+0000","last_unstale":"2024-09-11T22:12:13.206504+0000","last_undegraded":"2024-09-11T22:12:13.206504+0000","last_fullsized":"2024-09-11T22:12:13.206504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.006995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406452+0000","last_change":"2024-09-11T22:12:02.762553+0000","last_active":"2024-09-11T22:12:13.406452+0000","last_peered":"2024-09-11T22:12:13.406452+0000","last_clean":"2024-09-11T22:12:13.406452+0000","last_became_active":"2024-09-11T22:12:02.762197+0000","last_became_peered":"2024-09-11T22:12:02.762197+0000","last_unstale":"2024-09-11T22:12:13.406452+0000","last_undegraded":"2024-09-11T22:12:13.406452+0000","last_fullsized":"2024-09-11T22:12:13.406452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:07.188949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:38.143029+0000","last_change":"2024-09-11T22:12:00.817959+0000","last_active":"2024-09-11T22:12:38.143029+0000","last_peered":"2024-09-11T22:12:38.143029+0000","last_clean":"2024-09-11T22:12:38.143029+0000","last_became_active":"2024-09-11T22:12:00.817615+0000","last_became_peered":"2024-09-11T22:12:00.817615+0000","last_unstale":"2024-09-11T22:12:38.143029+0000","last_undegraded":"2024-09-11T22:12:38.143029+0000","last_fullsized":"2024-09-11T22:12:38.143029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:33.671580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406310+0000","last_change":"2024-09-11T22:12:09.811248+0000","last_active":"2024-09-11T22:12:13.406310+0000","last_peered":"2024-09-11T22:12:13.406310+0000","last_clean":"2024-09-11T22:12:13.406310+0000","last_became_active":"2024-09-11T22:12:09.810795+0000","last_became_peered":"2024-09-11T22:12:09.810795+0000","last_unstale":"2024-09-11T22:12:13.406310+0000","last_undegraded":"2024-09-11T22:12:13.406310+0000","last_fullsized":"2024-09-11T22:12:13.406310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:28.076656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406391+0000","last_change":"2024-09-11T22:12:02.773369+0000","last_active":"2024-09-11T22:12:13.406391+0000","last_peered":"2024-09-11T22:12:13.406391+0000","last_clean":"2024-09-11T22:12:13.406391+0000","last_became_active":"2024-09-11T22:12:02.773046+0000","last_became_peered":"2024-09-11T22:12:02.773046+0000","last_unstale":"2024-09-11T22:12:13.406391+0000","last_undegraded":"2024-09-11T22:12:13.406391+0000","last_fullsized":"2024-09-11T22:12:13.406391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:04.993273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.153680+0000","last_change":"2024-09-11T22:12:00.751696+0000","last_active":"2024-09-11T22:12:39.153680+0000","last_peered":"2024-09-11T22:12:39.153680+0000","last_clean":"2024-09-11T22:12:39.153680+0000","last_became_active":"2024-09-11T22:12:00.751362+0000","last_became_peered":"2024-09-11T22:12:00.751362+0000","last_unstale":"2024-09-11T22:12:39.153680+0000","last_undegraded":"2024-09-11T22:12:39.153680+0000","last_fullsized":"2024-09-11T22:12:39.153680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:19.372942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206343+0000","last_change":"2024-09-11T22:12:09.814219+0000","last_active":"2024-09-11T22:12:13.206343+0000","last_peered":"2024-09-11T22:12:13.206343+0000","last_clean":"2024-09-11T22:12:13.206343+0000","last_became_active":"2024-09-11T22:12:09.813817+0000","last_became_peered":"2024-09-11T22:12:09.813817+0000","last_unstale":"2024-09-11T22:12:13.206343+0000","last_undegraded":"2024-09-11T22:12:13.206343+0000","last_fullsized":"2024-09-11T22:12:13.206343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:36.473171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406648+0000","last_change":"2024-09-11T22:12:02.769739+0000","last_active":"2024-09-11T22:12:13.406648+0000","last_peered":"2024-09-11T22:12:13.406648+0000","last_clean":"2024-09-11T22:12:13.406648+0000","last_became_active":"2024-09-11T22:12:02.769118+0000","last_became_peered":"2024-09-11T22:12:02.769118+0000","last_unstale":"2024-09-11T22:12:13.406648+0000","last_undegraded":"2024-09-11T22:12:13.406648+0000","last_fullsized":"2024-09-11T22:12:13.406648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:01.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:49.172316+0000","last_change":"2024-09-11T22:12:00.748294+0000","last_active":"2024-09-11T22:12:49.172316+0000","last_peered":"2024-09-11T22:12:49.172316+0000","last_clean":"2024-09-11T22:12:49.172316+0000","last_became_active":"2024-09-11T22:12:00.748000+0000","last_became_peered":"2024-09-11T22:12:00.748000+0000","last_unstale":"2024-09-11T22:12:49.172316+0000","last_undegraded":"2024-09-11T22:12:49.172316+0000","last_fullsized":"2024-09-11T22:12:49.172316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:08.629756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406288+0000","last_change":"2024-09-11T22:12:09.820051+0000","last_active":"2024-09-11T22:12:13.406288+0000","last_peered":"2024-09-11T22:12:13.406288+0000","last_clean":"2024-09-11T22:12:13.406288+0000","last_became_active":"2024-09-11T22:12:09.819655+0000","last_became_peered":"2024-09-11T22:12:09.819655+0000","last_unstale":"2024-09-11T22:12:13.406288+0000","last_undegraded":"2024-09-11T22:12:13.406288+0000","last_fullsized":"2024-09-11T22:12:13.406288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:26.560012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405631+0000","last_change":"2024-09-11T22:12:09.812991+0000","last_active":"2024-09-11T22:12:13.405631+0000","last_peered":"2024-09-11T22:12:13.405631+0000","last_clean":"2024-09-11T22:12:13.405631+0000","last_became_active":"2024-09-11T22:12:09.812602+0000","last_became_peered":"2024-09-11T22:12:09.812602+0000","last_unstale":"2024-09-11T22:12:13.405631+0000","last_undegraded":"2024-09-11T22:12:13.405631+0000","last_fullsized":"2024-09-11T22:12:13.405631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:21.198192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206434+0000","last_change":"2024-09-11T22:12:02.760545+0000","last_active":"2024-09-11T22:12:13.206434+0000","last_peered":"2024-09-11T22:12:13.206434+0000","last_clean":"2024-09-11T22:12:13.206434+0000","last_became_active":"2024-09-11T22:12:02.760061+0000","last_became_peered":"2024-09-11T22:12:02.760061+0000","last_unstale":"2024-09-11T22:12:13.206434+0000","last_undegraded":"2024-09-11T22:12:13.206434+0000","last_fullsized":"2024-09-11T22:12:13.206434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:37.000552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.154239+0000","last_change":"2024-09-11T22:12:00.750643+0000","last_active":"2024-09-11T22:12:39.154239+0000","last_peered":"2024-09-11T22:12:39.154239+0000","last_clean":"2024-09-11T22:12:39.154239+0000","last_became_active":"2024-09-11T22:12:00.750129+0000","last_became_peered":"2024-09-11T22:12:00.750129+0000","last_unstale":"2024-09-11T22:12:39.154239+0000","last_undegraded":"2024-09-11T22:12:39.154239+0000","last_fullsized":"2024-09-11T22:12:39.154239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:25.519952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699394+0000","last_change":"2024-09-11T22:12:09.813607+0000","last_active":"2024-09-11T22:12:13.699394+0000","last_peered":"2024-09-11T22:12:13.699394+0000","last_clean":"2024-09-11T22:12:13.699394+0000","last_became_active":"2024-09-11T22:12:09.813201+0000","last_became_peered":"2024-09-11T22:12:09.813201+0000","last_unstale":"2024-09-11T22:12:13.699394+0000","last_undegraded":"2024-09-11T22:12:13.699394+0000","last_fullsized":"2024-09-11T22:12:13.699394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:39.793247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405813+0000","last_change":"2024-09-11T22:12:02.766428+0000","last_active":"2024-09-11T22:12:13.405813+0000","last_peered":"2024-09-11T22:12:13.405813+0000","last_clean":"2024-09-11T22:12:13.405813+0000","last_became_active":"2024-09-11T22:12:02.766112+0000","last_became_peered":"2024-09-11T22:12:02.766112+0000","last_unstale":"2024-09-11T22:12:13.405813+0000","last_undegraded":"2024-09-11T22:12:13.405813+0000","last_fullsized":"2024-09-11T22:12:13.405813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:24.701996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:37.132643+0000","last_change":"2024-09-11T22:12:00.743654+0000","last_active":"2024-09-11T22:12:37.132643+0000","last_peered":"2024-09-11T22:12:37.132643+0000","last_clean":"2024-09-11T22:12:37.132643+0000","last_became_active":"2024-09-11T22:12:00.743216+0000","last_became_peered":"2024-09-11T22:12:00.743216+0000","last_unstale":"2024-09-11T22:12:37.132643+0000","last_undegraded":"2024-09-11T22:12:37.132643+0000","last_fullsized":"2024-09-11T22:12:37.132643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.843451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:38.144295+0000","last_change":"2024-09-11T22:12:00.747738+0000","last_active":"2024-09-11T22:12:38.144295+0000","last_peered":"2024-09-11T22:12:38.144295+0000","last_clean":"2024-09-11T22:12:38.144295+0000","last_became_active":"2024-09-11T22:12:00.747424+0000","last_became_peered":"2024-09-11T22:12:00.747424+0000","last_unstale":"2024-09-11T22:12:38.144295+0000","last_undegraded":"2024-09-11T22:12:38.144295+0000","last_fullsized":"2024-09-11T22:12:38.144295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:30.097802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814750+0000","last_change":"2024-09-11T22:12:02.770033+0000","last_active":"2024-09-11T22:12:12.814750+0000","last_peered":"2024-09-11T22:12:12.814750+0000","last_clean":"2024-09-11T22:12:12.814750+0000","last_became_active":"2024-09-11T22:12:02.769597+0000","last_became_peered":"2024-09-11T22:12:02.769597+0000","last_unstale":"2024-09-11T22:12:12.814750+0000","last_undegraded":"2024-09-11T22:12:12.814750+0000","last_fullsized":"2024-09-11T22:12:12.814750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:35.447477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405926+0000","last_change":"2024-09-11T22:12:09.816652+0000","last_active":"2024-09-11T22:12:13.405926+0000","last_peered":"2024-09-11T22:12:13.405926+0000","last_clean":"2024-09-11T22:12:13.405926+0000","last_became_active":"2024-09-11T22:12:09.816259+0000","last_became_peered":"2024-09-11T22:12:09.816259+0000","last_unstale":"2024-09-11T22:12:13.405926+0000","last_undegraded":"2024-09-11T22:12:13.405926+0000","last_fullsized":"2024-09-11T22:12:13.405926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:09.818890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.170643+0000","last_change":"2024-09-11T22:12:00.752118+0000","last_active":"2024-09-11T22:12:47.170643+0000","last_peered":"2024-09-11T22:12:47.170643+0000","last_clean":"2024-09-11T22:12:47.170643+0000","last_became_active":"2024-09-11T22:12:00.751757+0000","last_became_peered":"2024-09-11T22:12:00.751757+0000","last_unstale":"2024-09-11T22:12:47.170643+0000","last_undegraded":"2024-09-11T22:12:47.170643+0000","last_fullsized":"2024-09-11T22:12:47.170643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:28.139391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405858+0000","last_change":"2024-09-11T22:12:02.772836+0000","last_active":"2024-09-11T22:12:13.405858+0000","last_peered":"2024-09-11T22:12:13.405858+0000","last_clean":"2024-09-11T22:12:13.405858+0000","last_became_active":"2024-09-11T22:12:02.772528+0000","last_became_peered":"2024-09-11T22:12:02.772528+0000","last_unstale":"2024-09-11T22:12:13.405858+0000","last_undegraded":"2024-09-11T22:12:13.405858+0000","last_fullsized":"2024-09-11T22:12:13.405858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:05.148091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406294+0000","last_change":"2024-09-11T22:12:09.820138+0000","last_active":"2024-09-11T22:12:13.406294+0000","last_peered":"2024-09-11T22:12:13.406294+0000","last_clean":"2024-09-11T22:12:13.406294+0000","last_became_active":"2024-09-11T22:12:09.819428+0000","last_became_peered":"2024-09-11T22:12:09.819428+0000","last_unstale":"2024-09-11T22:12:13.406294+0000","last_undegraded":"2024-09-11T22:12:13.406294+0000","last_fullsized":"2024-09-11T22:12:13.406294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:10.958913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.698955+0000","last_change":"2024-09-11T22:12:02.772732+0000","last_active":"2024-09-11T22:12:13.698955+0000","last_peered":"2024-09-11T22:12:13.698955+0000","last_clean":"2024-09-11T22:12:13.698955+0000","last_became_active":"2024-09-11T22:12:02.772407+0000","last_became_peered":"2024-09-11T22:12:02.772407+0000","last_unstale":"2024-09-11T22:12:13.698955+0000","last_undegraded":"2024-09-11T22:12:13.698955+0000","last_fullsized":"2024-09-11T22:12:13.698955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:24.134356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406202+0000","last_change":"2024-09-11T22:12:02.775825+0000","last_active":"2024-09-11T22:12:13.406202+0000","last_peered":"2024-09-11T22:12:13.406202+0000","last_clean":"2024-09-11T22:12:13.406202+0000","last_became_active":"2024-09-11T22:12:02.775471+0000","last_became_peered":"2024-09-11T22:12:02.775471+0000","last_unstale":"2024-09-11T22:12:13.406202+0000","last_undegraded":"2024-09-11T22:12:13.406202+0000","last_fullsized":"2024-09-11T22:12:13.406202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:46.333390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405778+0000","last_change":"2024-09-11T22:12:02.764570+0000","last_active":"2024-09-11T22:12:13.405778+0000","last_peered":"2024-09-11T22:12:13.405778+0000","last_clean":"2024-09-11T22:12:13.405778+0000","last_became_active":"2024-09-11T22:12:02.764005+0000","last_became_peered":"2024-09-11T22:12:02.764005+0000","last_unstale":"2024-09-11T22:12:13.405778+0000","last_undegraded":"2024-09-11T22:12:13.405778+0000","last_fullsized":"2024-09-11T22:12:13.405778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:32.169886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405913+0000","last_change":"2024-09-11T22:12:02.769156+0000","last_active":"2024-09-11T22:12:13.405913+0000","last_peered":"2024-09-11T22:12:13.405913+0000","last_clean":"2024-09-11T22:12:13.405913+0000","last_became_active":"2024-09-11T22:12:02.768795+0000","last_became_peered":"2024-09-11T22:12:02.768795+0000","last_unstale":"2024-09-11T22:12:13.405913+0000","last_undegraded":"2024-09-11T22:12:13.405913+0000","last_fullsized":"2024-09-11T22:12:13.405913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:25.506739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814102+0000","last_change":"2024-09-11T22:12:02.763102+0000","last_active":"2024-09-11T22:12:12.814102+0000","last_peered":"2024-09-11T22:12:12.814102+0000","last_clean":"2024-09-11T22:12:12.814102+0000","last_became_active":"2024-09-11T22:12:02.762687+0000","last_became_peered":"2024-09-11T22:12:02.762687+0000","last_unstale":"2024-09-11T22:12:12.814102+0000","last_undegraded":"2024-09-11T22:12:12.814102+0000","last_fullsized":"2024-09-11T22:12:12.814102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:05.069744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699307+0000","last_change":"2024-09-11T22:12:02.776229+0000","last_active":"2024-09-11T22:12:13.699307+0000","last_peered":"2024-09-11T22:12:13.699307+0000","last_clean":"2024-09-11T22:12:13.699307+0000","last_became_active":"2024-09-11T22:12:02.775875+0000","last_became_peered":"2024-09-11T22:12:02.775875+0000","last_unstale":"2024-09-11T22:12:13.699307+0000","last_undegraded":"2024-09-11T22:12:13.699307+0000","last_fullsized":"2024-09-11T22:12:13.699307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:46.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406451+0000","last_change":"2024-09-11T22:12:02.766783+0000","last_active":"2024-09-11T22:12:13.406451+0000","last_peered":"2024-09-11T22:12:13.406451+0000","last_clean":"2024-09-11T22:12:13.406451+0000","last_became_active":"2024-09-11T22:12:02.766280+0000","last_became_peered":"2024-09-11T22:12:02.766280+0000","last_unstale":"2024-09-11T22:12:13.406451+0000","last_undegraded":"2024-09-11T22:12:13.406451+0000","last_fullsized":"2024-09-11T22:12:13.406451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:12.657682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405985+0000","last_change":"2024-09-11T22:12:02.759008+0000","last_active":"2024-09-11T22:12:13.405985+0000","last_peered":"2024-09-11T22:12:13.405985+0000","last_clean":"2024-09-11T22:12:13.405985+0000","last_became_active":"2024-09-11T22:12:02.758734+0000","last_became_peered":"2024-09-11T22:12:02.758734+0000","last_unstale":"2024-09-11T22:12:13.405985+0000","last_undegraded":"2024-09-11T22:12:13.405985+0000","last_fullsized":"2024-09-11T22:12:13.405985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:37.729891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814400+0000","last_change":"2024-09-11T22:12:02.777260+0000","last_active":"2024-09-11T22:12:12.814400+0000","last_peered":"2024-09-11T22:12:12.814400+0000","last_clean":"2024-09-11T22:12:12.814400+0000","last_became_active":"2024-09-11T22:12:02.775486+0000","last_became_peered":"2024-09-11T22:12:02.775486+0000","last_unstale":"2024-09-11T22:12:12.814400+0000","last_undegraded":"2024-09-11T22:12:12.814400+0000","last_fullsized":"2024-09-11T22:12:12.814400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:41.099229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406324+0000","last_change":"2024-09-11T22:12:02.762025+0000","last_active":"2024-09-11T22:12:13.406324+0000","last_peered":"2024-09-11T22:12:13.406324+0000","last_clean":"2024-09-11T22:12:13.406324+0000","last_became_active":"2024-09-11T22:12:02.761751+0000","last_became_peered":"2024-09-11T22:12:02.761751+0000","last_unstale":"2024-09-11T22:12:13.406324+0000","last_undegraded":"2024-09-11T22:12:13.406324+0000","last_fullsized":"2024-09-11T22:12:13.406324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:45.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406579+0000","last_change":"2024-09-11T22:12:02.765001+0000","last_active":"2024-09-11T22:12:13.406579+0000","last_peered":"2024-09-11T22:12:13.406579+0000","last_clean":"2024-09-11T22:12:13.406579+0000","last_became_active":"2024-09-11T22:12:02.764667+0000","last_became_peered":"2024-09-11T22:12:02.764667+0000","last_unstale":"2024-09-11T22:12:13.406579+0000","last_undegraded":"2024-09-11T22:12:13.406579+0000","last_fullsized":"2024-09-11T22:12:13.406579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:59.074616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813910+0000","last_change":"2024-09-11T22:12:02.762972+0000","last_active":"2024-09-11T22:12:12.813910+0000","last_peered":"2024-09-11T22:12:12.813910+0000","last_clean":"2024-09-11T22:12:12.813910+0000","last_became_active":"2024-09-11T22:12:02.762685+0000","last_became_peered":"2024-09-11T22:12:02.762685+0000","last_unstale":"2024-09-11T22:12:12.813910+0000","last_undegraded":"2024-09-11T22:12:12.813910+0000","last_fullsized":"2024-09-11T22:12:12.813910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:34.278584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699093+0000","last_change":"2024-09-11T22:12:02.776351+0000","last_active":"2024-09-11T22:12:13.699093+0000","last_peered":"2024-09-11T22:12:13.699093+0000","last_clean":"2024-09-11T22:12:13.699093+0000","last_became_active":"2024-09-11T22:12:02.774827+0000","last_became_peered":"2024-09-11T22:12:02.774827+0000","last_unstale":"2024-09-11T22:12:13.699093+0000","last_undegraded":"2024-09-11T22:12:13.699093+0000","last_fullsized":"2024-09-11T22:12:13.699093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:55.887697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406408+0000","last_change":"2024-09-11T22:12:02.757412+0000","last_active":"2024-09-11T22:12:13.406408+0000","last_peered":"2024-09-11T22:12:13.406408+0000","last_clean":"2024-09-11T22:12:13.406408+0000","last_became_active":"2024-09-11T22:12:02.757013+0000","last_became_peered":"2024-09-11T22:12:02.757013+0000","last_unstale":"2024-09-11T22:12:13.406408+0000","last_undegraded":"2024-09-11T22:12:13.406408+0000","last_fullsized":"2024-09-11T22:12:13.406408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:21.468082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206836+0000","last_change":"2024-09-11T22:12:02.765943+0000","last_active":"2024-09-11T22:12:13.206836+0000","last_peered":"2024-09-11T22:12:13.206836+0000","last_clean":"2024-09-11T22:12:13.206836+0000","last_became_active":"2024-09-11T22:12:02.765617+0000","last_became_peered":"2024-09-11T22:12:02.765617+0000","last_unstale":"2024-09-11T22:12:13.206836+0000","last_undegraded":"2024-09-11T22:12:13.206836+0000","last_fullsized":"2024-09-11T22:12:13.206836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:47.023509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699544+0000","last_change":"2024-09-11T22:12:02.772379+0000","last_active":"2024-09-11T22:12:13.699544+0000","last_peered":"2024-09-11T22:12:13.699544+0000","last_clean":"2024-09-11T22:12:13.699544+0000","last_became_active":"2024-09-11T22:12:02.771561+0000","last_became_peered":"2024-09-11T22:12:02.771561+0000","last_unstale":"2024-09-11T22:12:13.699544+0000","last_undegraded":"2024-09-11T22:12:13.699544+0000","last_fullsized":"2024-09-11T22:12:13.699544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:52.988744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405677+0000","last_change":"2024-09-11T22:12:02.763489+0000","last_active":"2024-09-11T22:12:13.405677+0000","last_peered":"2024-09-11T22:12:13.405677+0000","last_clean":"2024-09-11T22:12:13.405677+0000","last_became_active":"2024-09-11T22:12:02.763194+0000","last_became_peered":"2024-09-11T22:12:02.763194+0000","last_unstale":"2024-09-11T22:12:13.405677+0000","last_undegraded":"2024-09-11T22:12:13.405677+0000","last_fullsized":"2024-09-11T22:12:13.405677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:15:30.685816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405436+0000","last_change":"2024-09-11T22:12:02.768537+0000","last_active":"2024-09-11T22:12:13.405436+0000","last_peered":"2024-09-11T22:12:13.405436+0000","last_clean":"2024-09-11T22:12:13.405436+0000","last_became_active":"2024-09-11T22:12:02.768219+0000","last_became_peered":"2024-09-11T22:12:02.768219+0000","last_unstale":"2024-09-11T22:12:13.405436+0000","last_undegraded":"2024-09-11T22:12:13.405436+0000","last_fullsized":"2024-09-11T22:12:13.405436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:45:03.177587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405569+0000","last_change":"2024-09-11T22:12:02.761823+0000","last_active":"2024-09-11T22:12:13.405569+0000","last_peered":"2024-09-11T22:12:13.405569+0000","last_clean":"2024-09-11T22:12:13.405569+0000","last_became_active":"2024-09-11T22:12:02.761435+0000","last_became_peered":"2024-09-11T22:12:02.761435+0000","last_unstale":"2024-09-11T22:12:13.405569+0000","last_undegraded":"2024-09-11T22:12:13.405569+0000","last_fullsized":"2024-09-11T22:12:13.405569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:54.535178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813827+0000","last_change":"2024-09-11T22:12:02.775613+0000","last_active":"2024-09-11T22:12:12.813827+0000","last_peered":"2024-09-11T22:12:12.813827+0000","last_clean":"2024-09-11T22:12:12.813827+0000","last_became_active":"2024-09-11T22:12:02.775383+0000","last_became_peered":"2024-09-11T22:12:02.775383+0000","last_unstale":"2024-09-11T22:12:12.813827+0000","last_undegraded":"2024-09-11T22:12:12.813827+0000","last_fullsized":"2024-09-11T22:12:12.813827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:18.973047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.407044+0000","last_change":"2024-09-11T22:12:02.771155+0000","last_active":"2024-09-11T22:12:13.407044+0000","last_peered":"2024-09-11T22:12:13.407044+0000","last_clean":"2024-09-11T22:12:13.407044+0000","last_became_active":"2024-09-11T22:12:02.770278+0000","last_became_peered":"2024-09-11T22:12:02.770278+0000","last_unstale":"2024-09-11T22:12:13.407044+0000","last_undegraded":"2024-09-11T22:12:13.407044+0000","last_fullsized":"2024-09-11T22:12:13.407044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:23.888372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814524+0000","last_change":"2024-09-11T22:12:02.772840+0000","last_active":"2024-09-11T22:12:12.814524+0000","last_peered":"2024-09-11T22:12:12.814524+0000","last_clean":"2024-09-11T22:12:12.814524+0000","last_became_active":"2024-09-11T22:12:02.772384+0000","last_became_peered":"2024-09-11T22:12:02.772384+0000","last_unstale":"2024-09-11T22:12:12.814524+0000","last_undegraded":"2024-09-11T22:12:12.814524+0000","last_fullsized":"2024-09-11T22:12:12.814524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:42.287294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.407568+0000","last_change":"2024-09-11T22:12:02.767741+0000","last_active":"2024-09-11T22:12:13.407568+0000","last_peered":"2024-09-11T22:12:13.407568+0000","last_clean":"2024-09-11T22:12:13.407568+0000","last_became_active":"2024-09-11T22:12:02.767234+0000","last_became_peered":"2024-09-11T22:12:02.767234+0000","last_unstale":"2024-09-11T22:12:13.407568+0000","last_undegraded":"2024-09-11T22:12:13.407568+0000","last_fullsized":"2024-09-11T22:12:13.407568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:22.250763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406966+0000","last_change":"2024-09-11T22:12:02.771265+0000","last_active":"2024-09-11T22:12:13.406966+0000","last_peered":"2024-09-11T22:12:13.406966+0000","last_clean":"2024-09-11T22:12:13.406966+0000","last_became_active":"2024-09-11T22:12:02.770617+0000","last_became_peered":"2024-09-11T22:12:02.770617+0000","last_unstale":"2024-09-11T22:12:13.406966+0000","last_undegraded":"2024-09-11T22:12:13.406966+0000","last_fullsized":"2024-09-11T22:12:13.406966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.390581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405722+0000","last_change":"2024-09-11T22:12:02.770850+0000","last_active":"2024-09-11T22:12:13.405722+0000","last_peered":"2024-09-11T22:12:13.405722+0000","last_clean":"2024-09-11T22:12:13.405722+0000","last_became_active":"2024-09-11T22:12:02.770559+0000","last_became_peered":"2024-09-11T22:12:02.770559+0000","last_unstale":"2024-09-11T22:12:13.405722+0000","last_undegraded":"2024-09-11T22:12:13.405722+0000","last_fullsized":"2024-09-11T22:12:13.405722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:06.865962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814107+0000","last_change":"2024-09-11T22:12:02.775947+0000","last_active":"2024-09-11T22:12:12.814107+0000","last_peered":"2024-09-11T22:12:12.814107+0000","last_clean":"2024-09-11T22:12:12.814107+0000","last_became_active":"2024-09-11T22:12:02.775559+0000","last_became_peered":"2024-09-11T22:12:02.775559+0000","last_unstale":"2024-09-11T22:12:12.814107+0000","last_undegraded":"2024-09-11T22:12:12.814107+0000","last_fullsized":"2024-09-11T22:12:12.814107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:14.642600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406122+0000","last_change":"2024-09-11T22:12:02.768124+0000","last_active":"2024-09-11T22:12:13.406122+0000","last_peered":"2024-09-11T22:12:13.406122+0000","last_clean":"2024-09-11T22:12:13.406122+0000","last_became_active":"2024-09-11T22:12:02.767804+0000","last_became_peered":"2024-09-11T22:12:02.767804+0000","last_unstale":"2024-09-11T22:12:13.406122+0000","last_undegraded":"2024-09-11T22:12:13.406122+0000","last_fullsized":"2024-09-11T22:12:13.406122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:23.612445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406414+0000","last_change":"2024-09-11T22:12:02.764086+0000","last_active":"2024-09-11T22:12:13.406414+0000","last_peered":"2024-09-11T22:12:13.406414+0000","last_clean":"2024-09-11T22:12:13.406414+0000","last_became_active":"2024-09-11T22:12:02.763708+0000","last_became_peered":"2024-09-11T22:12:02.763708+0000","last_unstale":"2024-09-11T22:12:13.406414+0000","last_undegraded":"2024-09-11T22:12:13.406414+0000","last_fullsized":"2024-09-11T22:12:13.406414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:06.058314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406319+0000","last_change":"2024-09-11T22:12:02.770820+0000","last_active":"2024-09-11T22:12:13.406319+0000","last_peered":"2024-09-11T22:12:13.406319+0000","last_clean":"2024-09-11T22:12:13.406319+0000","last_became_active":"2024-09-11T22:12:02.767911+0000","last_became_peered":"2024-09-11T22:12:02.767911+0000","last_unstale":"2024-09-11T22:12:13.406319+0000","last_undegraded":"2024-09-11T22:12:13.406319+0000","last_fullsized":"2024-09-11T22:12:13.406319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:12:04.030699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405899+0000","last_change":"2024-09-11T22:12:02.764702+0000","last_active":"2024-09-11T22:12:13.405899+0000","last_peered":"2024-09-11T22:12:13.405899+0000","last_clean":"2024-09-11T22:12:13.405899+0000","last_became_active":"2024-09-11T22:12:02.764407+0000","last_became_peered":"2024-09-11T22:12:02.764407+0000","last_unstale":"2024-09-11T22:12:13.405899+0000","last_undegraded":"2024-09-11T22:12:13.405899+0000","last_fullsized":"2024-09-11T22:12:13.405899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:01.551823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405722+0000","last_change":"2024-09-11T22:12:02.762209+0000","last_active":"2024-09-11T22:12:13.405722+0000","last_peered":"2024-09-11T22:12:13.405722+0000","last_clean":"2024-09-11T22:12:13.405722+0000","last_became_active":"2024-09-11T22:12:02.761936+0000","last_became_peered":"2024-09-11T22:12:02.761936+0000","last_unstale":"2024-09-11T22:12:13.405722+0000","last_undegraded":"2024-09-11T22:12:13.405722+0000","last_fullsized":"2024-09-11T22:12:13.405722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:39.667680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699536+0000","last_change":"2024-09-11T22:12:02.773537+0000","last_active":"2024-09-11T22:12:13.699536+0000","last_peered":"2024-09-11T22:12:13.699536+0000","last_clean":"2024-09-11T22:12:13.699536+0000","last_became_active":"2024-09-11T22:12:02.773145+0000","last_became_peered":"2024-09-11T22:12:02.773145+0000","last_unstale":"2024-09-11T22:12:13.699536+0000","last_undegraded":"2024-09-11T22:12:13.699536+0000","last_fullsized":"2024-09-11T22:12:13.699536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:30.058536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56306,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":233,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":163165,"data_compressed":5631,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286124,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449109,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":491369,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1127676,"data_compressed":55903,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":495395,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":491677,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":1088569,"data_compressed":54026,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462958,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":487196,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691891,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1083365,"data_compressed":54026,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855417,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":491270,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117054,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":486511,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":537151,"data_compressed":19524,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":531571,"data_compressed":19524,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":488259,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":45490,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51070,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":51227,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:12:54.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:12:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:54.882 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:54 smithi092 ceph-mon[31173]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:12:54.882 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:54 smithi092 ceph-mon[31173]: pgmap v309: 129 pgs: 129 active+clean; 632 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-11T22:12:54.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 -- 172.21.15.92:0/2354094275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480100eb0 msgr2=0x7fb480106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 --2- 172.21.15.92:0/2354094275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480100eb0 0x7fb480106840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb474009ca0 tx=0x7fb47402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 -- 172.21.15.92:0/2354094275 shutdown_connections 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 --2- 172.21.15.92:0/2354094275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb480106f70 0x7fb48010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 --2- 172.21.15.92:0/2354094275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480100eb0 0x7fb480106840 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 --2- 172.21.15.92:0/2354094275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb480100500 0x7fb4801008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 -- 172.21.15.92:0/2354094275 >> 172.21.15.92:0/2354094275 conn(0x7fb4800fc5c0 msgr2=0x7fb4800fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 -- 172.21.15.92:0/2354094275 shutdown_connections 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.960+0000 7fb484845640 1 -- 172.21.15.92:0/2354094275 wait complete. 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 Processor -- start 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 -- start start 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb480100500 0x7fb480101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb480100eb0 0x7fb4801019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480106f70 0x7fb480101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111e40 con 0x7fb480106f70 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111cc0 con 0x7fb480100eb0 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb484845640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111fc0 con 0x7fb480100500 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb47ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480106f70 0x7fb480101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb47ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480106f70 0x7fb480101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48010/0 (socket says 172.21.15.92:48010) 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.961+0000 7fb47ed9d640 1 -- 172.21.15.92:0/3238501589 learned_addr learned my addr 172.21.15.92:0/3238501589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47dd9b640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb480100eb0 0x7fb4801019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47e59c640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb480100500 0x7fb480101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47ed9d640 1 -- 172.21.15.92:0/3238501589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb480100500 msgr2=0x7fb480101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47ed9d640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb480100500 0x7fb480101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47ed9d640 1 -- 172.21.15.92:0/3238501589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb480100eb0 msgr2=0x7fb4801019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47ed9d640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb480100eb0 0x7fb4801019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47ed9d640 1 -- 172.21.15.92:0/3238501589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4801a67e0 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47dd9b640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb480100eb0 0x7fb4801019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb47ed9d640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480106f70 0x7fb480101f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c970 tx=0x7fb47000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb46f7fe640 1 -- 172.21.15.92:0/3238501589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470007bb0 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb46f7fe640 1 -- 172.21.15.92:0/3238501589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb470007d50 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4801a6ad0 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb46f7fe640 1 -- 172.21.15.92:0/3238501589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470018700 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.962+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4801a6f90 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.963+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4801008e0 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.964+0000 7fb46f7fe640 1 -- 172.21.15.92:0/3238501589 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb47000fa20 con 0x7fb480106f70 2024-09-11T22:12:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.965+0000 7fb46f7fe640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb4680a2c10 0x7fb4680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.965+0000 7fb47e59c640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb4680a2c10 0x7fb4680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.965+0000 7fb46f7fe640 1 -- 172.21.15.92:0/3238501589 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb4700cf480 con 0x7fb480106f70 2024-09-11T22:12:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.965+0000 7fb47e59c640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb4680a2c10 0x7fb4680a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb480069fc0 tx=0x7fb47800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:54.969+0000 7fb46f7fe640 1 -- 172.21.15.92:0/3238501589 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb470014030 con 0x7fb480106f70 2024-09-11T22:12:55.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:54 smithi153 ceph-mon[35045]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:12:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:54 smithi153 ceph-mon[35045]: pgmap v309: 129 pgs: 129 active+clean; 632 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-11T22:12:55.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.134+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb48010a150 con 0x7fb4680a2c10 2024-09-11T22:12:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.143+0000 7fb46f7fe640 1 -- 172.21.15.92:0/3238501589 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340702 (secure 0 0 0) 0x7fb478010be0 con 0x7fb4680a2c10 2024-09-11T22:12:55.147 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T22:12:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb4680a2c10 msgr2=0x7fb4680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb4680a2c10 0x7fb4680a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb480069fc0 tx=0x7fb47800b390 comp rx=0 tx=0).stop 2024-09-11T22:12:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480106f70 msgr2=0x7fb480101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480106f70 0x7fb480101f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c970 tx=0x7fb47000ce30 comp rx=0 tx=0).stop 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 shutdown_connections 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb4680a2c10 0x7fb4680a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb480106f70 0x7fb480101f00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb480100eb0 0x7fb4801019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 --2- 172.21.15.92:0/3238501589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb480100500 0x7fb480101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 >> 172.21.15.92:0/3238501589 conn(0x7fb4800fc5c0 msgr2=0x7fb4801052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 shutdown_connections 2024-09-11T22:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:55.146+0000 7fb484845640 1 -- 172.21.15.92:0/3238501589 wait complete. 2024-09-11T22:12:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:54 smithi110 ceph-mon[34830]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:12:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:54 smithi110 ceph-mon[34830]: pgmap v309: 129 pgs: 129 active+clean; 632 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-11T22:12:55.773 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-11T22:12:53.693454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646908,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":277,"num_read_kb":240,"num_write":416,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382064,"kb_used_data":12016,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559760,"statfs":{"total":1151940427776,"available":1151549194240,"internally_reserved":0,"allocated":12304384,"data_stored":7813413,"data_compressed":326532,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47030,"num_objects":20,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":151,"num_read_kb":131,"num_write":63,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.1f","version":"90'13","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.169054+0000","last_change":"2024-09-11T22:12:00.751331+0000","last_active":"2024-09-11T22:12:47.169054+0000","last_peered":"2024-09-11T22:12:47.169054+0000","last_clean":"2024-09-11T22:12:47.169054+0000","last_became_active":"2024-09-11T22:12:00.751034+0000","last_became_peered":"2024-09-11T22:12:00.751034+0000","last_unstale":"2024-09-11T22:12:47.169054+0000","last_undegraded":"2024-09-11T22:12:47.169054+0000","last_fullsized":"2024-09-11T22:12:47.169054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:14.418754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405924+0000","last_change":"2024-09-11T22:12:02.776052+0000","last_active":"2024-09-11T22:12:13.405924+0000","last_peered":"2024-09-11T22:12:13.405924+0000","last_clean":"2024-09-11T22:12:13.405924+0000","last_became_active":"2024-09-11T22:12:02.775727+0000","last_became_peered":"2024-09-11T22:12:02.775727+0000","last_unstale":"2024-09-11T22:12:13.405924+0000","last_undegraded":"2024-09-11T22:12:13.405924+0000","last_fullsized":"2024-09-11T22:12:13.405924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:50.805895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406006+0000","last_change":"2024-09-11T22:12:09.818054+0000","last_active":"2024-09-11T22:12:13.406006+0000","last_peered":"2024-09-11T22:12:13.406006+0000","last_clean":"2024-09-11T22:12:13.406006+0000","last_became_active":"2024-09-11T22:12:09.817527+0000","last_became_peered":"2024-09-11T22:12:09.817527+0000","last_unstale":"2024-09-11T22:12:13.406006+0000","last_undegraded":"2024-09-11T22:12:13.406006+0000","last_fullsized":"2024-09-11T22:12:13.406006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:26.496214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406116+0000","last_change":"2024-09-11T22:12:00.817451+0000","last_active":"2024-09-11T22:12:13.406116+0000","last_peered":"2024-09-11T22:12:13.406116+0000","last_clean":"2024-09-11T22:12:13.406116+0000","last_became_active":"2024-09-11T22:12:00.817047+0000","last_became_peered":"2024-09-11T22:12:00.817047+0000","last_unstale":"2024-09-11T22:12:13.406116+0000","last_undegraded":"2024-09-11T22:12:13.406116+0000","last_fullsized":"2024-09-11T22:12:13.406116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:48.450520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406368+0000","last_change":"2024-09-11T22:12:02.764638+0000","last_active":"2024-09-11T22:12:13.406368+0000","last_peered":"2024-09-11T22:12:13.406368+0000","last_clean":"2024-09-11T22:12:13.406368+0000","last_became_active":"2024-09-11T22:12:02.764329+0000","last_became_peered":"2024-09-11T22:12:02.764329+0000","last_unstale":"2024-09-11T22:12:13.406368+0000","last_undegraded":"2024-09-11T22:12:13.406368+0000","last_fullsized":"2024-09-11T22:12:13.406368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:27.322976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405814+0000","last_change":"2024-09-11T22:12:09.817478+0000","last_active":"2024-09-11T22:12:13.405814+0000","last_peered":"2024-09-11T22:12:13.405814+0000","last_clean":"2024-09-11T22:12:13.405814+0000","last_became_active":"2024-09-11T22:12:09.817098+0000","last_became_peered":"2024-09-11T22:12:09.817098+0000","last_unstale":"2024-09-11T22:12:13.405814+0000","last_undegraded":"2024-09-11T22:12:13.405814+0000","last_fullsized":"2024-09-11T22:12:13.405814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:50.066572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:40.149294+0000","last_change":"2024-09-11T22:12:00.754280+0000","last_active":"2024-09-11T22:12:40.149294+0000","last_peered":"2024-09-11T22:12:40.149294+0000","last_clean":"2024-09-11T22:12:40.149294+0000","last_became_active":"2024-09-11T22:12:00.753906+0000","last_became_peered":"2024-09-11T22:12:00.753906+0000","last_unstale":"2024-09-11T22:12:40.149294+0000","last_undegraded":"2024-09-11T22:12:40.149294+0000","last_fullsized":"2024-09-11T22:12:40.149294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:01.049033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406363+0000","last_change":"2024-09-11T22:12:02.765037+0000","last_active":"2024-09-11T22:12:13.406363+0000","last_peered":"2024-09-11T22:12:13.406363+0000","last_clean":"2024-09-11T22:12:13.406363+0000","last_became_active":"2024-09-11T22:12:02.764730+0000","last_became_peered":"2024-09-11T22:12:02.764730+0000","last_unstale":"2024-09-11T22:12:13.406363+0000","last_undegraded":"2024-09-11T22:12:13.406363+0000","last_fullsized":"2024-09-11T22:12:13.406363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:13.974389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406695+0000","last_change":"2024-09-11T22:12:09.817801+0000","last_active":"2024-09-11T22:12:13.406695+0000","last_peered":"2024-09-11T22:12:13.406695+0000","last_clean":"2024-09-11T22:12:13.406695+0000","last_became_active":"2024-09-11T22:12:09.817389+0000","last_became_peered":"2024-09-11T22:12:09.817389+0000","last_unstale":"2024-09-11T22:12:13.406695+0000","last_undegraded":"2024-09-11T22:12:13.406695+0000","last_fullsized":"2024-09-11T22:12:13.406695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:55.152113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.155140+0000","last_change":"2024-09-11T22:12:00.752216+0000","last_active":"2024-09-11T22:12:39.155140+0000","last_peered":"2024-09-11T22:12:39.155140+0000","last_clean":"2024-09-11T22:12:39.155140+0000","last_became_active":"2024-09-11T22:12:00.751641+0000","last_became_peered":"2024-09-11T22:12:00.751641+0000","last_unstale":"2024-09-11T22:12:39.155140+0000","last_undegraded":"2024-09-11T22:12:39.155140+0000","last_fullsized":"2024-09-11T22:12:39.155140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814969+0000","last_change":"2024-09-11T22:12:02.776766+0000","last_active":"2024-09-11T22:12:12.814969+0000","last_peered":"2024-09-11T22:12:12.814969+0000","last_clean":"2024-09-11T22:12:12.814969+0000","last_became_active":"2024-09-11T22:12:02.775985+0000","last_became_peered":"2024-09-11T22:12:02.775985+0000","last_unstale":"2024-09-11T22:12:12.814969+0000","last_undegraded":"2024-09-11T22:12:12.814969+0000","last_fullsized":"2024-09-11T22:12:12.814969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:39.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405815+0000","last_change":"2024-09-11T22:12:09.817906+0000","last_active":"2024-09-11T22:12:13.405815+0000","last_peered":"2024-09-11T22:12:13.405815+0000","last_clean":"2024-09-11T22:12:13.405815+0000","last_became_active":"2024-09-11T22:12:09.817404+0000","last_became_peered":"2024-09-11T22:12:09.817404+0000","last_unstale":"2024-09-11T22:12:13.405815+0000","last_undegraded":"2024-09-11T22:12:13.405815+0000","last_fullsized":"2024-09-11T22:12:13.405815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:17:29.957890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814132+0000","last_change":"2024-09-11T22:12:02.770932+0000","last_active":"2024-09-11T22:12:12.814132+0000","last_peered":"2024-09-11T22:12:12.814132+0000","last_clean":"2024-09-11T22:12:12.814132+0000","last_became_active":"2024-09-11T22:12:02.770619+0000","last_became_peered":"2024-09-11T22:12:02.770619+0000","last_unstale":"2024-09-11T22:12:12.814132+0000","last_undegraded":"2024-09-11T22:12:12.814132+0000","last_fullsized":"2024-09-11T22:12:12.814132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:31.746892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:36.129600+0000","last_change":"2024-09-11T22:12:00.754437+0000","last_active":"2024-09-11T22:12:36.129600+0000","last_peered":"2024-09-11T22:12:36.129600+0000","last_clean":"2024-09-11T22:12:36.129600+0000","last_became_active":"2024-09-11T22:12:00.753874+0000","last_became_peered":"2024-09-11T22:12:00.753874+0000","last_unstale":"2024-09-11T22:12:36.129600+0000","last_undegraded":"2024-09-11T22:12:36.129600+0000","last_fullsized":"2024-09-11T22:12:36.129600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:49.305272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406481+0000","last_change":"2024-09-11T22:12:10.199589+0000","last_active":"2024-09-11T22:12:13.406481+0000","last_peered":"2024-09-11T22:12:13.406481+0000","last_clean":"2024-09-11T22:12:13.406481+0000","last_became_active":"2024-09-11T22:12:10.199185+0000","last_became_peered":"2024-09-11T22:12:10.199185+0000","last_unstale":"2024-09-11T22:12:13.406481+0000","last_undegraded":"2024-09-11T22:12:13.406481+0000","last_fullsized":"2024-09-11T22:12:13.406481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:51.081016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405566+0000","last_change":"2024-09-11T22:12:02.763355+0000","last_active":"2024-09-11T22:12:13.405566+0000","last_peered":"2024-09-11T22:12:13.405566+0000","last_clean":"2024-09-11T22:12:13.405566+0000","last_became_active":"2024-09-11T22:12:02.763060+0000","last_became_peered":"2024-09-11T22:12:02.763060+0000","last_unstale":"2024-09-11T22:12:13.405566+0000","last_undegraded":"2024-09-11T22:12:13.405566+0000","last_fullsized":"2024-09-11T22:12:13.405566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:22.776503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:38.143624+0000","last_change":"2024-09-11T22:12:00.754158+0000","last_active":"2024-09-11T22:12:38.143624+0000","last_peered":"2024-09-11T22:12:38.143624+0000","last_clean":"2024-09-11T22:12:38.143624+0000","last_became_active":"2024-09-11T22:12:00.753706+0000","last_became_peered":"2024-09-11T22:12:00.753706+0000","last_unstale":"2024-09-11T22:12:38.143624+0000","last_undegraded":"2024-09-11T22:12:38.143624+0000","last_fullsized":"2024-09-11T22:12:38.143624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:40.470704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406214+0000","last_change":"2024-09-11T22:12:09.810374+0000","last_active":"2024-09-11T22:12:13.406214+0000","last_peered":"2024-09-11T22:12:13.406214+0000","last_clean":"2024-09-11T22:12:13.406214+0000","last_became_active":"2024-09-11T22:12:09.809915+0000","last_became_peered":"2024-09-11T22:12:09.809915+0000","last_unstale":"2024-09-11T22:12:13.406214+0000","last_undegraded":"2024-09-11T22:12:13.406214+0000","last_fullsized":"2024-09-11T22:12:13.406214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:25.554920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699035+0000","last_change":"2024-09-11T22:12:02.769350+0000","last_active":"2024-09-11T22:12:13.699035+0000","last_peered":"2024-09-11T22:12:13.699035+0000","last_clean":"2024-09-11T22:12:13.699035+0000","last_became_active":"2024-09-11T22:12:02.768896+0000","last_became_peered":"2024-09-11T22:12:02.768896+0000","last_unstale":"2024-09-11T22:12:13.699035+0000","last_undegraded":"2024-09-11T22:12:13.699035+0000","last_fullsized":"2024-09-11T22:12:13.699035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:12.490938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.150797+0000","last_change":"2024-09-11T22:12:00.743281+0000","last_active":"2024-09-11T22:12:39.150797+0000","last_peered":"2024-09-11T22:12:39.150797+0000","last_clean":"2024-09-11T22:12:39.150797+0000","last_became_active":"2024-09-11T22:12:00.742977+0000","last_became_peered":"2024-09-11T22:12:00.742977+0000","last_unstale":"2024-09-11T22:12:39.150797+0000","last_undegraded":"2024-09-11T22:12:39.150797+0000","last_fullsized":"2024-09-11T22:12:39.150797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:28.890328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206889+0000","last_change":"2024-09-11T22:12:09.815471+0000","last_active":"2024-09-11T22:12:13.206889+0000","last_peered":"2024-09-11T22:12:13.206889+0000","last_clean":"2024-09-11T22:12:13.206889+0000","last_became_active":"2024-09-11T22:12:09.815098+0000","last_became_peered":"2024-09-11T22:12:09.815098+0000","last_unstale":"2024-09-11T22:12:13.206889+0000","last_undegraded":"2024-09-11T22:12:13.206889+0000","last_fullsized":"2024-09-11T22:12:13.206889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:36.375848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405512+0000","last_change":"2024-09-11T22:12:02.763597+0000","last_active":"2024-09-11T22:12:13.405512+0000","last_peered":"2024-09-11T22:12:13.405512+0000","last_clean":"2024-09-11T22:12:13.405512+0000","last_became_active":"2024-09-11T22:12:02.763242+0000","last_became_peered":"2024-09-11T22:12:02.763242+0000","last_unstale":"2024-09-11T22:12:13.405512+0000","last_undegraded":"2024-09-11T22:12:13.405512+0000","last_fullsized":"2024-09-11T22:12:13.405512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:24.968799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:36.128741+0000","last_change":"2024-09-11T22:12:00.817604+0000","last_active":"2024-09-11T22:12:36.128741+0000","last_peered":"2024-09-11T22:12:36.128741+0000","last_clean":"2024-09-11T22:12:36.128741+0000","last_became_active":"2024-09-11T22:12:00.817377+0000","last_became_peered":"2024-09-11T22:12:00.817377+0000","last_unstale":"2024-09-11T22:12:36.128741+0000","last_undegraded":"2024-09-11T22:12:36.128741+0000","last_fullsized":"2024-09-11T22:12:36.128741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:27.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406696+0000","last_change":"2024-09-11T22:12:10.201074+0000","last_active":"2024-09-11T22:12:13.406696+0000","last_peered":"2024-09-11T22:12:13.406696+0000","last_clean":"2024-09-11T22:12:13.406696+0000","last_became_active":"2024-09-11T22:12:10.200664+0000","last_became_peered":"2024-09-11T22:12:10.200664+0000","last_unstale":"2024-09-11T22:12:13.406696+0000","last_undegraded":"2024-09-11T22:12:13.406696+0000","last_fullsized":"2024-09-11T22:12:13.406696+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:44.784751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406118+0000","last_change":"2024-09-11T22:12:02.773380+0000","last_active":"2024-09-11T22:12:13.406118+0000","last_peered":"2024-09-11T22:12:13.406118+0000","last_clean":"2024-09-11T22:12:13.406118+0000","last_became_active":"2024-09-11T22:12:02.773076+0000","last_became_peered":"2024-09-11T22:12:02.773076+0000","last_unstale":"2024-09-11T22:12:13.406118+0000","last_undegraded":"2024-09-11T22:12:13.406118+0000","last_fullsized":"2024-09-11T22:12:13.406118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:35.475581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.152455+0000","last_change":"2024-09-11T22:12:00.746438+0000","last_active":"2024-09-11T22:12:39.152455+0000","last_peered":"2024-09-11T22:12:39.152455+0000","last_clean":"2024-09-11T22:12:39.152455+0000","last_became_active":"2024-09-11T22:12:00.746034+0000","last_became_peered":"2024-09-11T22:12:00.746034+0000","last_unstale":"2024-09-11T22:12:39.152455+0000","last_undegraded":"2024-09-11T22:12:39.152455+0000","last_fullsized":"2024-09-11T22:12:39.152455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:49.156748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406088+0000","last_change":"2024-09-11T22:12:09.812553+0000","last_active":"2024-09-11T22:12:13.406088+0000","last_peered":"2024-09-11T22:12:13.406088+0000","last_clean":"2024-09-11T22:12:13.406088+0000","last_became_active":"2024-09-11T22:12:09.812092+0000","last_became_peered":"2024-09-11T22:12:09.812092+0000","last_unstale":"2024-09-11T22:12:13.406088+0000","last_undegraded":"2024-09-11T22:12:13.406088+0000","last_fullsized":"2024-09-11T22:12:13.406088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:13.947799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406006+0000","last_change":"2024-09-11T22:12:02.763399+0000","last_active":"2024-09-11T22:12:13.406006+0000","last_peered":"2024-09-11T22:12:13.406006+0000","last_clean":"2024-09-11T22:12:13.406006+0000","last_became_active":"2024-09-11T22:12:02.763082+0000","last_became_peered":"2024-09-11T22:12:02.763082+0000","last_unstale":"2024-09-11T22:12:13.406006+0000","last_undegraded":"2024-09-11T22:12:13.406006+0000","last_fullsized":"2024-09-11T22:12:13.406006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:52.267960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.776645+0000","last_change":"2024-09-11T22:12:00.751318+0000","last_active":"2024-09-11T22:12:47.776645+0000","last_peered":"2024-09-11T22:12:47.776645+0000","last_clean":"2024-09-11T22:12:47.776645+0000","last_became_active":"2024-09-11T22:12:00.751022+0000","last_became_peered":"2024-09-11T22:12:00.751022+0000","last_unstale":"2024-09-11T22:12:47.776645+0000","last_undegraded":"2024-09-11T22:12:47.776645+0000","last_fullsized":"2024-09-11T22:12:47.776645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:23:26.163502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405558+0000","last_change":"2024-09-11T22:12:10.200279+0000","last_active":"2024-09-11T22:12:13.405558+0000","last_peered":"2024-09-11T22:12:13.405558+0000","last_clean":"2024-09-11T22:12:13.405558+0000","last_became_active":"2024-09-11T22:12:10.199920+0000","last_became_peered":"2024-09-11T22:12:10.199920+0000","last_unstale":"2024-09-11T22:12:13.405558+0000","last_undegraded":"2024-09-11T22:12:13.405558+0000","last_fullsized":"2024-09-11T22:12:13.405558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:59.012735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814148+0000","last_change":"2024-09-11T22:12:02.755565+0000","last_active":"2024-09-11T22:12:12.814148+0000","last_peered":"2024-09-11T22:12:12.814148+0000","last_clean":"2024-09-11T22:12:12.814148+0000","last_became_active":"2024-09-11T22:12:02.755236+0000","last_became_peered":"2024-09-11T22:12:02.755236+0000","last_unstale":"2024-09-11T22:12:12.814148+0000","last_undegraded":"2024-09-11T22:12:12.814148+0000","last_fullsized":"2024-09-11T22:12:12.814148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.588976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.154310+0000","last_change":"2024-09-11T22:12:00.818634+0000","last_active":"2024-09-11T22:12:39.154310+0000","last_peered":"2024-09-11T22:12:39.154310+0000","last_clean":"2024-09-11T22:12:39.154310+0000","last_became_active":"2024-09-11T22:12:00.817917+0000","last_became_peered":"2024-09-11T22:12:00.817917+0000","last_unstale":"2024-09-11T22:12:39.154310+0000","last_undegraded":"2024-09-11T22:12:39.154310+0000","last_fullsized":"2024-09-11T22:12:39.154310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:42.382962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699203+0000","last_change":"2024-09-11T22:12:09.814770+0000","last_active":"2024-09-11T22:12:13.699203+0000","last_peered":"2024-09-11T22:12:13.699203+0000","last_clean":"2024-09-11T22:12:13.699203+0000","last_became_active":"2024-09-11T22:12:09.814322+0000","last_became_peered":"2024-09-11T22:12:09.814322+0000","last_unstale":"2024-09-11T22:12:13.699203+0000","last_undegraded":"2024-09-11T22:12:13.699203+0000","last_fullsized":"2024-09-11T22:12:13.699203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.666161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.407233+0000","last_change":"2024-09-11T22:12:02.762939+0000","last_active":"2024-09-11T22:12:13.407233+0000","last_peered":"2024-09-11T22:12:13.407233+0000","last_clean":"2024-09-11T22:12:13.407233+0000","last_became_active":"2024-09-11T22:12:02.762579+0000","last_became_peered":"2024-09-11T22:12:02.762579+0000","last_unstale":"2024-09-11T22:12:13.407233+0000","last_undegraded":"2024-09-11T22:12:13.407233+0000","last_fullsized":"2024-09-11T22:12:13.407233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:35.124612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206779+0000","last_change":"2024-09-11T22:12:00.818094+0000","last_active":"2024-09-11T22:12:13.206779+0000","last_peered":"2024-09-11T22:12:13.206779+0000","last_clean":"2024-09-11T22:12:13.206779+0000","last_became_active":"2024-09-11T22:12:00.817723+0000","last_became_peered":"2024-09-11T22:12:00.817723+0000","last_unstale":"2024-09-11T22:12:13.206779+0000","last_undegraded":"2024-09-11T22:12:13.206779+0000","last_fullsized":"2024-09-11T22:12:13.206779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:32.618689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405603+0000","last_change":"2024-09-11T22:12:10.199778+0000","last_active":"2024-09-11T22:12:13.405603+0000","last_peered":"2024-09-11T22:12:13.405603+0000","last_clean":"2024-09-11T22:12:13.405603+0000","last_became_active":"2024-09-11T22:12:10.199280+0000","last_became_peered":"2024-09-11T22:12:10.199280+0000","last_unstale":"2024-09-11T22:12:13.405603+0000","last_undegraded":"2024-09-11T22:12:13.405603+0000","last_fullsized":"2024-09-11T22:12:13.405603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:08.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406572+0000","last_change":"2024-09-11T22:12:02.769477+0000","last_active":"2024-09-11T22:12:13.406572+0000","last_peered":"2024-09-11T22:12:13.406572+0000","last_clean":"2024-09-11T22:12:13.406572+0000","last_became_active":"2024-09-11T22:12:02.768847+0000","last_became_peered":"2024-09-11T22:12:02.768847+0000","last_unstale":"2024-09-11T22:12:13.406572+0000","last_undegraded":"2024-09-11T22:12:13.406572+0000","last_fullsized":"2024-09-11T22:12:13.406572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:57.612115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:52.175323+0000","last_change":"2024-09-11T22:12:00.744559+0000","last_active":"2024-09-11T22:12:52.175323+0000","last_peered":"2024-09-11T22:12:52.175323+0000","last_clean":"2024-09-11T22:12:52.175323+0000","last_became_active":"2024-09-11T22:12:00.744266+0000","last_became_peered":"2024-09-11T22:12:00.744266+0000","last_unstale":"2024-09-11T22:12:52.175323+0000","last_undegraded":"2024-09-11T22:12:52.175323+0000","last_fullsized":"2024-09-11T22:12:52.175323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:12.663275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814107+0000","last_change":"2024-09-11T22:12:09.819627+0000","last_active":"2024-09-11T22:12:12.814107+0000","last_peered":"2024-09-11T22:12:12.814107+0000","last_clean":"2024-09-11T22:12:12.814107+0000","last_became_active":"2024-09-11T22:12:09.819241+0000","last_became_peered":"2024-09-11T22:12:09.819241+0000","last_unstale":"2024-09-11T22:12:12.814107+0000","last_undegraded":"2024-09-11T22:12:12.814107+0000","last_fullsized":"2024-09-11T22:12:12.814107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:32.932384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814449+0000","last_change":"2024-09-11T22:12:02.776860+0000","last_active":"2024-09-11T22:12:12.814449+0000","last_peered":"2024-09-11T22:12:12.814449+0000","last_clean":"2024-09-11T22:12:12.814449+0000","last_became_active":"2024-09-11T22:12:02.776421+0000","last_became_peered":"2024-09-11T22:12:02.776421+0000","last_unstale":"2024-09-11T22:12:12.814449+0000","last_undegraded":"2024-09-11T22:12:12.814449+0000","last_fullsized":"2024-09-11T22:12:12.814449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:23:43.022309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:47.169340+0000","last_change":"2024-09-11T22:12:00.817499+0000","last_active":"2024-09-11T22:12:47.169340+0000","last_peered":"2024-09-11T22:12:47.169340+0000","last_clean":"2024-09-11T22:12:47.169340+0000","last_became_active":"2024-09-11T22:12:00.817152+0000","last_became_peered":"2024-09-11T22:12:00.817152+0000","last_unstale":"2024-09-11T22:12:47.169340+0000","last_undegraded":"2024-09-11T22:12:47.169340+0000","last_fullsized":"2024-09-11T22:12:47.169340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:19.716267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813879+0000","last_change":"2024-09-11T22:12:10.200318+0000","last_active":"2024-09-11T22:12:12.813879+0000","last_peered":"2024-09-11T22:12:12.813879+0000","last_clean":"2024-09-11T22:12:12.813879+0000","last_became_active":"2024-09-11T22:12:10.199904+0000","last_became_peered":"2024-09-11T22:12:10.199904+0000","last_unstale":"2024-09-11T22:12:12.813879+0000","last_undegraded":"2024-09-11T22:12:12.813879+0000","last_fullsized":"2024-09-11T22:12:12.813879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:47.061489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814023+0000","last_change":"2024-09-11T22:12:02.758872+0000","last_active":"2024-09-11T22:12:12.814023+0000","last_peered":"2024-09-11T22:12:12.814023+0000","last_clean":"2024-09-11T22:12:12.814023+0000","last_became_active":"2024-09-11T22:12:02.758578+0000","last_became_peered":"2024-09-11T22:12:02.758578+0000","last_unstale":"2024-09-11T22:12:12.814023+0000","last_undegraded":"2024-09-11T22:12:12.814023+0000","last_fullsized":"2024-09-11T22:12:12.814023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:51.875760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:37.134811+0000","last_change":"2024-09-11T22:12:00.749202+0000","last_active":"2024-09-11T22:12:37.134811+0000","last_peered":"2024-09-11T22:12:37.134811+0000","last_clean":"2024-09-11T22:12:37.134811+0000","last_became_active":"2024-09-11T22:12:00.748886+0000","last_became_peered":"2024-09-11T22:12:00.748886+0000","last_unstale":"2024-09-11T22:12:37.134811+0000","last_undegraded":"2024-09-11T22:12:37.134811+0000","last_fullsized":"2024-09-11T22:12:37.134811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:44.581466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.815165+0000","last_change":"2024-09-11T22:12:10.201018+0000","last_active":"2024-09-11T22:12:12.815165+0000","last_peered":"2024-09-11T22:12:12.815165+0000","last_clean":"2024-09-11T22:12:12.815165+0000","last_became_active":"2024-09-11T22:12:10.200658+0000","last_became_peered":"2024-09-11T22:12:10.200658+0000","last_unstale":"2024-09-11T22:12:12.815165+0000","last_undegraded":"2024-09-11T22:12:12.815165+0000","last_fullsized":"2024-09-11T22:12:12.815165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.919652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405760+0000","last_change":"2024-09-11T22:12:02.771546+0000","last_active":"2024-09-11T22:12:13.405760+0000","last_peered":"2024-09-11T22:12:13.405760+0000","last_clean":"2024-09-11T22:12:13.405760+0000","last_became_active":"2024-09-11T22:12:02.771296+0000","last_became_peered":"2024-09-11T22:12:02.771296+0000","last_unstale":"2024-09-11T22:12:13.405760+0000","last_undegraded":"2024-09-11T22:12:13.405760+0000","last_fullsized":"2024-09-11T22:12:13.405760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:05.144166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:40.156378+0000","last_change":"2024-09-11T22:12:00.744431+0000","last_active":"2024-09-11T22:12:40.156378+0000","last_peered":"2024-09-11T22:12:40.156378+0000","last_clean":"2024-09-11T22:12:40.156378+0000","last_became_active":"2024-09-11T22:12:00.744121+0000","last_became_peered":"2024-09-11T22:12:00.744121+0000","last_unstale":"2024-09-11T22:12:40.156378+0000","last_undegraded":"2024-09-11T22:12:40.156378+0000","last_fullsized":"2024-09-11T22:12:40.156378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:16.832914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814329+0000","last_change":"2024-09-11T22:10:51.241028+0000","last_active":"2024-09-11T22:12:12.814329+0000","last_peered":"2024-09-11T22:12:12.814329+0000","last_clean":"2024-09-11T22:12:12.814329+0000","last_became_active":"2024-09-11T22:10:51.240533+0000","last_became_peered":"2024-09-11T22:10:51.240533+0000","last_unstale":"2024-09-11T22:12:12.814329+0000","last_undegraded":"2024-09-11T22:12:12.814329+0000","last_fullsized":"2024-09-11T22:12:12.814329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_clean_scrub_stamp":"2024-09-11T22:07:30.402427+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:59.522909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.465088+0000","last_change":"2024-09-11T22:12:09.815386+0000","last_active":"2024-09-11T22:12:13.465088+0000","last_peered":"2024-09-11T22:12:13.465088+0000","last_clean":"2024-09-11T22:12:13.465088+0000","last_became_active":"2024-09-11T22:12:09.815004+0000","last_became_peered":"2024-09-11T22:12:09.815004+0000","last_unstale":"2024-09-11T22:12:13.465088+0000","last_undegraded":"2024-09-11T22:12:13.465088+0000","last_fullsized":"2024-09-11T22:12:13.465088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:22.247309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814644+0000","last_change":"2024-09-11T22:12:02.774139+0000","last_active":"2024-09-11T22:12:12.814644+0000","last_peered":"2024-09-11T22:12:12.814644+0000","last_clean":"2024-09-11T22:12:12.814644+0000","last_became_active":"2024-09-11T22:12:02.773785+0000","last_became_peered":"2024-09-11T22:12:02.773785+0000","last_unstale":"2024-09-11T22:12:12.814644+0000","last_undegraded":"2024-09-11T22:12:12.814644+0000","last_fullsized":"2024-09-11T22:12:12.814644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:53.160278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:52.176395+0000","last_change":"2024-09-11T22:12:00.818813+0000","last_active":"2024-09-11T22:12:52.176395+0000","last_peered":"2024-09-11T22:12:52.176395+0000","last_clean":"2024-09-11T22:12:52.176395+0000","last_became_active":"2024-09-11T22:12:00.817574+0000","last_became_peered":"2024-09-11T22:12:00.817574+0000","last_unstale":"2024-09-11T22:12:52.176395+0000","last_undegraded":"2024-09-11T22:12:52.176395+0000","last_fullsized":"2024-09-11T22:12:52.176395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:52.429428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406089+0000","last_change":"2024-09-11T22:12:09.819009+0000","last_active":"2024-09-11T22:12:13.406089+0000","last_peered":"2024-09-11T22:12:13.406089+0000","last_clean":"2024-09-11T22:12:13.406089+0000","last_became_active":"2024-09-11T22:12:09.818513+0000","last_became_peered":"2024-09-11T22:12:09.818513+0000","last_unstale":"2024-09-11T22:12:13.406089+0000","last_undegraded":"2024-09-11T22:12:13.406089+0000","last_fullsized":"2024-09-11T22:12:13.406089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:34.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814408+0000","last_change":"2024-09-11T22:12:02.776027+0000","last_active":"2024-09-11T22:12:12.814408+0000","last_peered":"2024-09-11T22:12:12.814408+0000","last_clean":"2024-09-11T22:12:12.814408+0000","last_became_active":"2024-09-11T22:12:02.775485+0000","last_became_peered":"2024-09-11T22:12:02.775485+0000","last_unstale":"2024-09-11T22:12:12.814408+0000","last_undegraded":"2024-09-11T22:12:12.814408+0000","last_fullsized":"2024-09-11T22:12:12.814408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:02.801205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.153793+0000","last_change":"2024-09-11T22:12:00.747291+0000","last_active":"2024-09-11T22:12:39.153793+0000","last_peered":"2024-09-11T22:12:39.153793+0000","last_clean":"2024-09-11T22:12:39.153793+0000","last_became_active":"2024-09-11T22:12:00.746910+0000","last_became_peered":"2024-09-11T22:12:00.746910+0000","last_unstale":"2024-09-11T22:12:39.153793+0000","last_undegraded":"2024-09-11T22:12:39.153793+0000","last_fullsized":"2024-09-11T22:12:39.153793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:23.993924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405817+0000","last_change":"2024-09-11T22:12:09.813999+0000","last_active":"2024-09-11T22:12:13.405817+0000","last_peered":"2024-09-11T22:12:13.405817+0000","last_clean":"2024-09-11T22:12:13.405817+0000","last_became_active":"2024-09-11T22:12:09.813626+0000","last_became_peered":"2024-09-11T22:12:09.813626+0000","last_unstale":"2024-09-11T22:12:13.405817+0000","last_undegraded":"2024-09-11T22:12:13.405817+0000","last_fullsized":"2024-09-11T22:12:13.405817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:12.069515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814677+0000","last_change":"2024-09-11T22:12:02.766451+0000","last_active":"2024-09-11T22:12:12.814677+0000","last_peered":"2024-09-11T22:12:12.814677+0000","last_clean":"2024-09-11T22:12:12.814677+0000","last_became_active":"2024-09-11T22:12:02.765909+0000","last_became_peered":"2024-09-11T22:12:02.765909+0000","last_unstale":"2024-09-11T22:12:12.814677+0000","last_undegraded":"2024-09-11T22:12:12.814677+0000","last_fullsized":"2024-09-11T22:12:12.814677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:34.099196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.151594+0000","last_change":"2024-09-11T22:12:00.748989+0000","last_active":"2024-09-11T22:12:39.151594+0000","last_peered":"2024-09-11T22:12:39.151594+0000","last_clean":"2024-09-11T22:12:39.151594+0000","last_became_active":"2024-09-11T22:12:00.748653+0000","last_became_peered":"2024-09-11T22:12:00.748653+0000","last_unstale":"2024-09-11T22:12:39.151594+0000","last_undegraded":"2024-09-11T22:12:39.151594+0000","last_fullsized":"2024-09-11T22:12:39.151594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:53.234719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814331+0000","last_change":"2024-09-11T22:12:09.818187+0000","last_active":"2024-09-11T22:12:12.814331+0000","last_peered":"2024-09-11T22:12:12.814331+0000","last_clean":"2024-09-11T22:12:12.814331+0000","last_became_active":"2024-09-11T22:12:09.817355+0000","last_became_peered":"2024-09-11T22:12:09.817355+0000","last_unstale":"2024-09-11T22:12:12.814331+0000","last_undegraded":"2024-09-11T22:12:12.814331+0000","last_fullsized":"2024-09-11T22:12:12.814331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:57.355716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699334+0000","last_change":"2024-09-11T22:12:02.774309+0000","last_active":"2024-09-11T22:12:13.699334+0000","last_peered":"2024-09-11T22:12:13.699334+0000","last_clean":"2024-09-11T22:12:13.699334+0000","last_became_active":"2024-09-11T22:12:02.774004+0000","last_became_peered":"2024-09-11T22:12:02.774004+0000","last_unstale":"2024-09-11T22:12:13.699334+0000","last_undegraded":"2024-09-11T22:12:13.699334+0000","last_fullsized":"2024-09-11T22:12:13.699334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:03.107880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.129039+0000","last_change":"2024-09-11T22:12:00.749564+0000","last_active":"2024-09-11T22:12:39.129039+0000","last_peered":"2024-09-11T22:12:39.129039+0000","last_clean":"2024-09-11T22:12:39.129039+0000","last_became_active":"2024-09-11T22:12:00.749221+0000","last_became_peered":"2024-09-11T22:12:00.749221+0000","last_unstale":"2024-09-11T22:12:39.129039+0000","last_undegraded":"2024-09-11T22:12:39.129039+0000","last_fullsized":"2024-09-11T22:12:39.129039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:42:10.875683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814553+0000","last_change":"2024-09-11T22:12:09.821945+0000","last_active":"2024-09-11T22:12:12.814553+0000","last_peered":"2024-09-11T22:12:12.814553+0000","last_clean":"2024-09-11T22:12:12.814553+0000","last_became_active":"2024-09-11T22:12:09.821537+0000","last_became_peered":"2024-09-11T22:12:09.821537+0000","last_unstale":"2024-09-11T22:12:12.814553+0000","last_undegraded":"2024-09-11T22:12:12.814553+0000","last_fullsized":"2024-09-11T22:12:12.814553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:18.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814459+0000","last_change":"2024-09-11T22:12:02.768511+0000","last_active":"2024-09-11T22:12:12.814459+0000","last_peered":"2024-09-11T22:12:12.814459+0000","last_clean":"2024-09-11T22:12:12.814459+0000","last_became_active":"2024-09-11T22:12:02.768060+0000","last_became_peered":"2024-09-11T22:12:02.768060+0000","last_unstale":"2024-09-11T22:12:12.814459+0000","last_undegraded":"2024-09-11T22:12:12.814459+0000","last_fullsized":"2024-09-11T22:12:12.814459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:26.417066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.153490+0000","last_change":"2024-09-11T22:12:00.817941+0000","last_active":"2024-09-11T22:12:39.153490+0000","last_peered":"2024-09-11T22:12:39.153490+0000","last_clean":"2024-09-11T22:12:39.153490+0000","last_became_active":"2024-09-11T22:12:00.817606+0000","last_became_peered":"2024-09-11T22:12:00.817606+0000","last_unstale":"2024-09-11T22:12:39.153490+0000","last_undegraded":"2024-09-11T22:12:39.153490+0000","last_fullsized":"2024-09-11T22:12:39.153490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:44.212961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814031+0000","last_change":"2024-09-11T22:12:09.812516+0000","last_active":"2024-09-11T22:12:12.814031+0000","last_peered":"2024-09-11T22:12:12.814031+0000","last_clean":"2024-09-11T22:12:12.814031+0000","last_became_active":"2024-09-11T22:12:09.812217+0000","last_became_peered":"2024-09-11T22:12:09.812217+0000","last_unstale":"2024-09-11T22:12:12.814031+0000","last_undegraded":"2024-09-11T22:12:12.814031+0000","last_fullsized":"2024-09-11T22:12:12.814031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:36.568803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813978+0000","last_change":"2024-09-11T22:12:02.775797+0000","last_active":"2024-09-11T22:12:12.813978+0000","last_peered":"2024-09-11T22:12:12.813978+0000","last_clean":"2024-09-11T22:12:12.813978+0000","last_became_active":"2024-09-11T22:12:02.775474+0000","last_became_peered":"2024-09-11T22:12:02.775474+0000","last_unstale":"2024-09-11T22:12:12.813978+0000","last_undegraded":"2024-09-11T22:12:12.813978+0000","last_fullsized":"2024-09-11T22:12:12.813978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:25.446633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:51.174695+0000","last_change":"2024-09-11T22:12:00.744237+0000","last_active":"2024-09-11T22:12:51.174695+0000","last_peered":"2024-09-11T22:12:51.174695+0000","last_clean":"2024-09-11T22:12:51.174695+0000","last_became_active":"2024-09-11T22:12:00.743783+0000","last_became_peered":"2024-09-11T22:12:00.743783+0000","last_unstale":"2024-09-11T22:12:51.174695+0000","last_undegraded":"2024-09-11T22:12:51.174695+0000","last_fullsized":"2024-09-11T22:12:51.174695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:53.046685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206953+0000","last_change":"2024-09-11T22:12:09.815279+0000","last_active":"2024-09-11T22:12:13.206953+0000","last_peered":"2024-09-11T22:12:13.206953+0000","last_clean":"2024-09-11T22:12:13.206953+0000","last_became_active":"2024-09-11T22:12:09.814447+0000","last_became_peered":"2024-09-11T22:12:09.814447+0000","last_unstale":"2024-09-11T22:12:13.206953+0000","last_undegraded":"2024-09-11T22:12:13.206953+0000","last_fullsized":"2024-09-11T22:12:13.206953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.289292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814217+0000","last_change":"2024-09-11T22:12:02.769205+0000","last_active":"2024-09-11T22:12:12.814217+0000","last_peered":"2024-09-11T22:12:12.814217+0000","last_clean":"2024-09-11T22:12:12.814217+0000","last_became_active":"2024-09-11T22:12:02.768890+0000","last_became_peered":"2024-09-11T22:12:02.768890+0000","last_unstale":"2024-09-11T22:12:12.814217+0000","last_undegraded":"2024-09-11T22:12:12.814217+0000","last_fullsized":"2024-09-11T22:12:12.814217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:43.705862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.151899+0000","last_change":"2024-09-11T22:12:00.749008+0000","last_active":"2024-09-11T22:12:39.151899+0000","last_peered":"2024-09-11T22:12:39.151899+0000","last_clean":"2024-09-11T22:12:39.151899+0000","last_became_active":"2024-09-11T22:12:00.748642+0000","last_became_peered":"2024-09-11T22:12:00.748642+0000","last_unstale":"2024-09-11T22:12:39.151899+0000","last_undegraded":"2024-09-11T22:12:39.151899+0000","last_fullsized":"2024-09-11T22:12:39.151899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:34.763101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699172+0000","last_change":"2024-09-11T22:12:09.816925+0000","last_active":"2024-09-11T22:12:13.699172+0000","last_peered":"2024-09-11T22:12:13.699172+0000","last_clean":"2024-09-11T22:12:13.699172+0000","last_became_active":"2024-09-11T22:12:09.816434+0000","last_became_peered":"2024-09-11T22:12:09.816434+0000","last_unstale":"2024-09-11T22:12:13.699172+0000","last_undegraded":"2024-09-11T22:12:13.699172+0000","last_fullsized":"2024-09-11T22:12:13.699172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813718+0000","last_change":"2024-09-11T22:12:02.761721+0000","last_active":"2024-09-11T22:12:12.813718+0000","last_peered":"2024-09-11T22:12:12.813718+0000","last_clean":"2024-09-11T22:12:12.813718+0000","last_became_active":"2024-09-11T22:12:02.761448+0000","last_became_peered":"2024-09-11T22:12:02.761448+0000","last_unstale":"2024-09-11T22:12:12.813718+0000","last_undegraded":"2024-09-11T22:12:12.813718+0000","last_fullsized":"2024-09-11T22:12:12.813718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:51.401729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:40.156667+0000","last_change":"2024-09-11T22:12:00.751110+0000","last_active":"2024-09-11T22:12:40.156667+0000","last_peered":"2024-09-11T22:12:40.156667+0000","last_clean":"2024-09-11T22:12:40.156667+0000","last_became_active":"2024-09-11T22:12:00.749431+0000","last_became_peered":"2024-09-11T22:12:00.749431+0000","last_unstale":"2024-09-11T22:12:40.156667+0000","last_undegraded":"2024-09-11T22:12:40.156667+0000","last_fullsized":"2024-09-11T22:12:40.156667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:27.774865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.815091+0000","last_change":"2024-09-11T22:12:09.820852+0000","last_active":"2024-09-11T22:12:12.815091+0000","last_peered":"2024-09-11T22:12:12.815091+0000","last_clean":"2024-09-11T22:12:12.815091+0000","last_became_active":"2024-09-11T22:12:09.820263+0000","last_became_peered":"2024-09-11T22:12:09.820263+0000","last_unstale":"2024-09-11T22:12:12.815091+0000","last_undegraded":"2024-09-11T22:12:12.815091+0000","last_fullsized":"2024-09-11T22:12:12.815091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:09.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406473+0000","last_change":"2024-09-11T22:12:02.770819+0000","last_active":"2024-09-11T22:12:13.406473+0000","last_peered":"2024-09-11T22:12:13.406473+0000","last_clean":"2024-09-11T22:12:13.406473+0000","last_became_active":"2024-09-11T22:12:02.770502+0000","last_became_peered":"2024-09-11T22:12:02.770502+0000","last_unstale":"2024-09-11T22:12:13.406473+0000","last_undegraded":"2024-09-11T22:12:13.406473+0000","last_fullsized":"2024-09-11T22:12:13.406473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:10.345880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'28","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:49.174220+0000","last_change":"2024-09-11T22:12:00.747739+0000","last_active":"2024-09-11T22:12:49.174220+0000","last_peered":"2024-09-11T22:12:49.174220+0000","last_clean":"2024-09-11T22:12:49.174220+0000","last_became_active":"2024-09-11T22:12:00.747372+0000","last_became_peered":"2024-09-11T22:12:00.747372+0000","last_unstale":"2024-09-11T22:12:49.174220+0000","last_undegraded":"2024-09-11T22:12:49.174220+0000","last_fullsized":"2024-09-11T22:12:49.174220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:12.943134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":30,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206504+0000","last_change":"2024-09-11T22:12:09.819643+0000","last_active":"2024-09-11T22:12:13.206504+0000","last_peered":"2024-09-11T22:12:13.206504+0000","last_clean":"2024-09-11T22:12:13.206504+0000","last_became_active":"2024-09-11T22:12:09.819144+0000","last_became_peered":"2024-09-11T22:12:09.819144+0000","last_unstale":"2024-09-11T22:12:13.206504+0000","last_undegraded":"2024-09-11T22:12:13.206504+0000","last_fullsized":"2024-09-11T22:12:13.206504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.006995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406452+0000","last_change":"2024-09-11T22:12:02.762553+0000","last_active":"2024-09-11T22:12:13.406452+0000","last_peered":"2024-09-11T22:12:13.406452+0000","last_clean":"2024-09-11T22:12:13.406452+0000","last_became_active":"2024-09-11T22:12:02.762197+0000","last_became_peered":"2024-09-11T22:12:02.762197+0000","last_unstale":"2024-09-11T22:12:13.406452+0000","last_undegraded":"2024-09-11T22:12:13.406452+0000","last_fullsized":"2024-09-11T22:12:13.406452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:07.188949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:38.143029+0000","last_change":"2024-09-11T22:12:00.817959+0000","last_active":"2024-09-11T22:12:38.143029+0000","last_peered":"2024-09-11T22:12:38.143029+0000","last_clean":"2024-09-11T22:12:38.143029+0000","last_became_active":"2024-09-11T22:12:00.817615+0000","last_became_peered":"2024-09-11T22:12:00.817615+0000","last_unstale":"2024-09-11T22:12:38.143029+0000","last_undegraded":"2024-09-11T22:12:38.143029+0000","last_fullsized":"2024-09-11T22:12:38.143029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:33.671580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406310+0000","last_change":"2024-09-11T22:12:09.811248+0000","last_active":"2024-09-11T22:12:13.406310+0000","last_peered":"2024-09-11T22:12:13.406310+0000","last_clean":"2024-09-11T22:12:13.406310+0000","last_became_active":"2024-09-11T22:12:09.810795+0000","last_became_peered":"2024-09-11T22:12:09.810795+0000","last_unstale":"2024-09-11T22:12:13.406310+0000","last_undegraded":"2024-09-11T22:12:13.406310+0000","last_fullsized":"2024-09-11T22:12:13.406310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:28.076656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406391+0000","last_change":"2024-09-11T22:12:02.773369+0000","last_active":"2024-09-11T22:12:13.406391+0000","last_peered":"2024-09-11T22:12:13.406391+0000","last_clean":"2024-09-11T22:12:13.406391+0000","last_became_active":"2024-09-11T22:12:02.773046+0000","last_became_peered":"2024-09-11T22:12:02.773046+0000","last_unstale":"2024-09-11T22:12:13.406391+0000","last_undegraded":"2024-09-11T22:12:13.406391+0000","last_fullsized":"2024-09-11T22:12:13.406391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:04.993273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.153680+0000","last_change":"2024-09-11T22:12:00.751696+0000","last_active":"2024-09-11T22:12:39.153680+0000","last_peered":"2024-09-11T22:12:39.153680+0000","last_clean":"2024-09-11T22:12:39.153680+0000","last_became_active":"2024-09-11T22:12:00.751362+0000","last_became_peered":"2024-09-11T22:12:00.751362+0000","last_unstale":"2024-09-11T22:12:39.153680+0000","last_undegraded":"2024-09-11T22:12:39.153680+0000","last_fullsized":"2024-09-11T22:12:39.153680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:19.372942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206343+0000","last_change":"2024-09-11T22:12:09.814219+0000","last_active":"2024-09-11T22:12:13.206343+0000","last_peered":"2024-09-11T22:12:13.206343+0000","last_clean":"2024-09-11T22:12:13.206343+0000","last_became_active":"2024-09-11T22:12:09.813817+0000","last_became_peered":"2024-09-11T22:12:09.813817+0000","last_unstale":"2024-09-11T22:12:13.206343+0000","last_undegraded":"2024-09-11T22:12:13.206343+0000","last_fullsized":"2024-09-11T22:12:13.206343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:36.473171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406648+0000","last_change":"2024-09-11T22:12:02.769739+0000","last_active":"2024-09-11T22:12:13.406648+0000","last_peered":"2024-09-11T22:12:13.406648+0000","last_clean":"2024-09-11T22:12:13.406648+0000","last_became_active":"2024-09-11T22:12:02.769118+0000","last_became_peered":"2024-09-11T22:12:02.769118+0000","last_unstale":"2024-09-11T22:12:13.406648+0000","last_undegraded":"2024-09-11T22:12:13.406648+0000","last_fullsized":"2024-09-11T22:12:13.406648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:01.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:49.172316+0000","last_change":"2024-09-11T22:12:00.748294+0000","last_active":"2024-09-11T22:12:49.172316+0000","last_peered":"2024-09-11T22:12:49.172316+0000","last_clean":"2024-09-11T22:12:49.172316+0000","last_became_active":"2024-09-11T22:12:00.748000+0000","last_became_peered":"2024-09-11T22:12:00.748000+0000","last_unstale":"2024-09-11T22:12:49.172316+0000","last_undegraded":"2024-09-11T22:12:49.172316+0000","last_fullsized":"2024-09-11T22:12:49.172316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:08.629756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406288+0000","last_change":"2024-09-11T22:12:09.820051+0000","last_active":"2024-09-11T22:12:13.406288+0000","last_peered":"2024-09-11T22:12:13.406288+0000","last_clean":"2024-09-11T22:12:13.406288+0000","last_became_active":"2024-09-11T22:12:09.819655+0000","last_became_peered":"2024-09-11T22:12:09.819655+0000","last_unstale":"2024-09-11T22:12:13.406288+0000","last_undegraded":"2024-09-11T22:12:13.406288+0000","last_fullsized":"2024-09-11T22:12:13.406288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:26.560012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405631+0000","last_change":"2024-09-11T22:12:09.812991+0000","last_active":"2024-09-11T22:12:13.405631+0000","last_peered":"2024-09-11T22:12:13.405631+0000","last_clean":"2024-09-11T22:12:13.405631+0000","last_became_active":"2024-09-11T22:12:09.812602+0000","last_became_peered":"2024-09-11T22:12:09.812602+0000","last_unstale":"2024-09-11T22:12:13.405631+0000","last_undegraded":"2024-09-11T22:12:13.405631+0000","last_fullsized":"2024-09-11T22:12:13.405631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:21.198192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206434+0000","last_change":"2024-09-11T22:12:02.760545+0000","last_active":"2024-09-11T22:12:13.206434+0000","last_peered":"2024-09-11T22:12:13.206434+0000","last_clean":"2024-09-11T22:12:13.206434+0000","last_became_active":"2024-09-11T22:12:02.760061+0000","last_became_peered":"2024-09-11T22:12:02.760061+0000","last_unstale":"2024-09-11T22:12:13.206434+0000","last_undegraded":"2024-09-11T22:12:13.206434+0000","last_fullsized":"2024-09-11T22:12:13.206434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:37.000552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:39.154239+0000","last_change":"2024-09-11T22:12:00.750643+0000","last_active":"2024-09-11T22:12:39.154239+0000","last_peered":"2024-09-11T22:12:39.154239+0000","last_clean":"2024-09-11T22:12:39.154239+0000","last_became_active":"2024-09-11T22:12:00.750129+0000","last_became_peered":"2024-09-11T22:12:00.750129+0000","last_unstale":"2024-09-11T22:12:39.154239+0000","last_undegraded":"2024-09-11T22:12:39.154239+0000","last_fullsized":"2024-09-11T22:12:39.154239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:25.519952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699394+0000","last_change":"2024-09-11T22:12:09.813607+0000","last_active":"2024-09-11T22:12:13.699394+0000","last_peered":"2024-09-11T22:12:13.699394+0000","last_clean":"2024-09-11T22:12:13.699394+0000","last_became_active":"2024-09-11T22:12:09.813201+0000","last_became_peered":"2024-09-11T22:12:09.813201+0000","last_unstale":"2024-09-11T22:12:13.699394+0000","last_undegraded":"2024-09-11T22:12:13.699394+0000","last_fullsized":"2024-09-11T22:12:13.699394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:39.793247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405813+0000","last_change":"2024-09-11T22:12:02.766428+0000","last_active":"2024-09-11T22:12:13.405813+0000","last_peered":"2024-09-11T22:12:13.405813+0000","last_clean":"2024-09-11T22:12:13.405813+0000","last_became_active":"2024-09-11T22:12:02.766112+0000","last_became_peered":"2024-09-11T22:12:02.766112+0000","last_unstale":"2024-09-11T22:12:13.405813+0000","last_undegraded":"2024-09-11T22:12:13.405813+0000","last_fullsized":"2024-09-11T22:12:13.405813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:24.701996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:37.132643+0000","last_change":"2024-09-11T22:12:00.743654+0000","last_active":"2024-09-11T22:12:37.132643+0000","last_peered":"2024-09-11T22:12:37.132643+0000","last_clean":"2024-09-11T22:12:37.132643+0000","last_became_active":"2024-09-11T22:12:00.743216+0000","last_became_peered":"2024-09-11T22:12:00.743216+0000","last_unstale":"2024-09-11T22:12:37.132643+0000","last_undegraded":"2024-09-11T22:12:37.132643+0000","last_fullsized":"2024-09-11T22:12:37.132643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.843451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:38.144295+0000","last_change":"2024-09-11T22:12:00.747738+0000","last_active":"2024-09-11T22:12:38.144295+0000","last_peered":"2024-09-11T22:12:38.144295+0000","last_clean":"2024-09-11T22:12:38.144295+0000","last_became_active":"2024-09-11T22:12:00.747424+0000","last_became_peered":"2024-09-11T22:12:00.747424+0000","last_unstale":"2024-09-11T22:12:38.144295+0000","last_undegraded":"2024-09-11T22:12:38.144295+0000","last_fullsized":"2024-09-11T22:12:38.144295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:30.097802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814750+0000","last_change":"2024-09-11T22:12:02.770033+0000","last_active":"2024-09-11T22:12:12.814750+0000","last_peered":"2024-09-11T22:12:12.814750+0000","last_clean":"2024-09-11T22:12:12.814750+0000","last_became_active":"2024-09-11T22:12:02.769597+0000","last_became_peered":"2024-09-11T22:12:02.769597+0000","last_unstale":"2024-09-11T22:12:12.814750+0000","last_undegraded":"2024-09-11T22:12:12.814750+0000","last_fullsized":"2024-09-11T22:12:12.814750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:35.447477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405926+0000","last_change":"2024-09-11T22:12:09.816652+0000","last_active":"2024-09-11T22:12:13.405926+0000","last_peered":"2024-09-11T22:12:13.405926+0000","last_clean":"2024-09-11T22:12:13.405926+0000","last_became_active":"2024-09-11T22:12:09.816259+0000","last_became_peered":"2024-09-11T22:12:09.816259+0000","last_unstale":"2024-09-11T22:12:13.405926+0000","last_undegraded":"2024-09-11T22:12:13.405926+0000","last_fullsized":"2024-09-11T22:12:13.405926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:09.818890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:52.176985+0000","last_change":"2024-09-11T22:12:00.752118+0000","last_active":"2024-09-11T22:12:52.176985+0000","last_peered":"2024-09-11T22:12:52.176985+0000","last_clean":"2024-09-11T22:12:52.176985+0000","last_became_active":"2024-09-11T22:12:00.751757+0000","last_became_peered":"2024-09-11T22:12:00.751757+0000","last_unstale":"2024-09-11T22:12:52.176985+0000","last_undegraded":"2024-09-11T22:12:52.176985+0000","last_fullsized":"2024-09-11T22:12:52.176985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:28.139391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405858+0000","last_change":"2024-09-11T22:12:02.772836+0000","last_active":"2024-09-11T22:12:13.405858+0000","last_peered":"2024-09-11T22:12:13.405858+0000","last_clean":"2024-09-11T22:12:13.405858+0000","last_became_active":"2024-09-11T22:12:02.772528+0000","last_became_peered":"2024-09-11T22:12:02.772528+0000","last_unstale":"2024-09-11T22:12:13.405858+0000","last_undegraded":"2024-09-11T22:12:13.405858+0000","last_fullsized":"2024-09-11T22:12:13.405858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:05.148091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406294+0000","last_change":"2024-09-11T22:12:09.820138+0000","last_active":"2024-09-11T22:12:13.406294+0000","last_peered":"2024-09-11T22:12:13.406294+0000","last_clean":"2024-09-11T22:12:13.406294+0000","last_became_active":"2024-09-11T22:12:09.819428+0000","last_became_peered":"2024-09-11T22:12:09.819428+0000","last_unstale":"2024-09-11T22:12:13.406294+0000","last_undegraded":"2024-09-11T22:12:13.406294+0000","last_fullsized":"2024-09-11T22:12:13.406294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:10.958913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.698955+0000","last_change":"2024-09-11T22:12:02.772732+0000","last_active":"2024-09-11T22:12:13.698955+0000","last_peered":"2024-09-11T22:12:13.698955+0000","last_clean":"2024-09-11T22:12:13.698955+0000","last_became_active":"2024-09-11T22:12:02.772407+0000","last_became_peered":"2024-09-11T22:12:02.772407+0000","last_unstale":"2024-09-11T22:12:13.698955+0000","last_undegraded":"2024-09-11T22:12:13.698955+0000","last_fullsized":"2024-09-11T22:12:13.698955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:24.134356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406202+0000","last_change":"2024-09-11T22:12:02.775825+0000","last_active":"2024-09-11T22:12:13.406202+0000","last_peered":"2024-09-11T22:12:13.406202+0000","last_clean":"2024-09-11T22:12:13.406202+0000","last_became_active":"2024-09-11T22:12:02.775471+0000","last_became_peered":"2024-09-11T22:12:02.775471+0000","last_unstale":"2024-09-11T22:12:13.406202+0000","last_undegraded":"2024-09-11T22:12:13.406202+0000","last_fullsized":"2024-09-11T22:12:13.406202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:46.333390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405778+0000","last_change":"2024-09-11T22:12:02.764570+0000","last_active":"2024-09-11T22:12:13.405778+0000","last_peered":"2024-09-11T22:12:13.405778+0000","last_clean":"2024-09-11T22:12:13.405778+0000","last_became_active":"2024-09-11T22:12:02.764005+0000","last_became_peered":"2024-09-11T22:12:02.764005+0000","last_unstale":"2024-09-11T22:12:13.405778+0000","last_undegraded":"2024-09-11T22:12:13.405778+0000","last_fullsized":"2024-09-11T22:12:13.405778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:32.169886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405913+0000","last_change":"2024-09-11T22:12:02.769156+0000","last_active":"2024-09-11T22:12:13.405913+0000","last_peered":"2024-09-11T22:12:13.405913+0000","last_clean":"2024-09-11T22:12:13.405913+0000","last_became_active":"2024-09-11T22:12:02.768795+0000","last_became_peered":"2024-09-11T22:12:02.768795+0000","last_unstale":"2024-09-11T22:12:13.405913+0000","last_undegraded":"2024-09-11T22:12:13.405913+0000","last_fullsized":"2024-09-11T22:12:13.405913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:25.506739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814102+0000","last_change":"2024-09-11T22:12:02.763102+0000","last_active":"2024-09-11T22:12:12.814102+0000","last_peered":"2024-09-11T22:12:12.814102+0000","last_clean":"2024-09-11T22:12:12.814102+0000","last_became_active":"2024-09-11T22:12:02.762687+0000","last_became_peered":"2024-09-11T22:12:02.762687+0000","last_unstale":"2024-09-11T22:12:12.814102+0000","last_undegraded":"2024-09-11T22:12:12.814102+0000","last_fullsized":"2024-09-11T22:12:12.814102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:05.069744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699307+0000","last_change":"2024-09-11T22:12:02.776229+0000","last_active":"2024-09-11T22:12:13.699307+0000","last_peered":"2024-09-11T22:12:13.699307+0000","last_clean":"2024-09-11T22:12:13.699307+0000","last_became_active":"2024-09-11T22:12:02.775875+0000","last_became_peered":"2024-09-11T22:12:02.775875+0000","last_unstale":"2024-09-11T22:12:13.699307+0000","last_undegraded":"2024-09-11T22:12:13.699307+0000","last_fullsized":"2024-09-11T22:12:13.699307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:46.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406451+0000","last_change":"2024-09-11T22:12:02.766783+0000","last_active":"2024-09-11T22:12:13.406451+0000","last_peered":"2024-09-11T22:12:13.406451+0000","last_clean":"2024-09-11T22:12:13.406451+0000","last_became_active":"2024-09-11T22:12:02.766280+0000","last_became_peered":"2024-09-11T22:12:02.766280+0000","last_unstale":"2024-09-11T22:12:13.406451+0000","last_undegraded":"2024-09-11T22:12:13.406451+0000","last_fullsized":"2024-09-11T22:12:13.406451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:12.657682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405985+0000","last_change":"2024-09-11T22:12:02.759008+0000","last_active":"2024-09-11T22:12:13.405985+0000","last_peered":"2024-09-11T22:12:13.405985+0000","last_clean":"2024-09-11T22:12:13.405985+0000","last_became_active":"2024-09-11T22:12:02.758734+0000","last_became_peered":"2024-09-11T22:12:02.758734+0000","last_unstale":"2024-09-11T22:12:13.405985+0000","last_undegraded":"2024-09-11T22:12:13.405985+0000","last_fullsized":"2024-09-11T22:12:13.405985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:37.729891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814400+0000","last_change":"2024-09-11T22:12:02.777260+0000","last_active":"2024-09-11T22:12:12.814400+0000","last_peered":"2024-09-11T22:12:12.814400+0000","last_clean":"2024-09-11T22:12:12.814400+0000","last_became_active":"2024-09-11T22:12:02.775486+0000","last_became_peered":"2024-09-11T22:12:02.775486+0000","last_unstale":"2024-09-11T22:12:12.814400+0000","last_undegraded":"2024-09-11T22:12:12.814400+0000","last_fullsized":"2024-09-11T22:12:12.814400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:41.099229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406324+0000","last_change":"2024-09-11T22:12:02.762025+0000","last_active":"2024-09-11T22:12:13.406324+0000","last_peered":"2024-09-11T22:12:13.406324+0000","last_clean":"2024-09-11T22:12:13.406324+0000","last_became_active":"2024-09-11T22:12:02.761751+0000","last_became_peered":"2024-09-11T22:12:02.761751+0000","last_unstale":"2024-09-11T22:12:13.406324+0000","last_undegraded":"2024-09-11T22:12:13.406324+0000","last_fullsized":"2024-09-11T22:12:13.406324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:45.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406579+0000","last_change":"2024-09-11T22:12:02.765001+0000","last_active":"2024-09-11T22:12:13.406579+0000","last_peered":"2024-09-11T22:12:13.406579+0000","last_clean":"2024-09-11T22:12:13.406579+0000","last_became_active":"2024-09-11T22:12:02.764667+0000","last_became_peered":"2024-09-11T22:12:02.764667+0000","last_unstale":"2024-09-11T22:12:13.406579+0000","last_undegraded":"2024-09-11T22:12:13.406579+0000","last_fullsized":"2024-09-11T22:12:13.406579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:59.074616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813910+0000","last_change":"2024-09-11T22:12:02.762972+0000","last_active":"2024-09-11T22:12:12.813910+0000","last_peered":"2024-09-11T22:12:12.813910+0000","last_clean":"2024-09-11T22:12:12.813910+0000","last_became_active":"2024-09-11T22:12:02.762685+0000","last_became_peered":"2024-09-11T22:12:02.762685+0000","last_unstale":"2024-09-11T22:12:12.813910+0000","last_undegraded":"2024-09-11T22:12:12.813910+0000","last_fullsized":"2024-09-11T22:12:12.813910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:34.278584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699093+0000","last_change":"2024-09-11T22:12:02.776351+0000","last_active":"2024-09-11T22:12:13.699093+0000","last_peered":"2024-09-11T22:12:13.699093+0000","last_clean":"2024-09-11T22:12:13.699093+0000","last_became_active":"2024-09-11T22:12:02.774827+0000","last_became_peered":"2024-09-11T22:12:02.774827+0000","last_unstale":"2024-09-11T22:12:13.699093+0000","last_undegraded":"2024-09-11T22:12:13.699093+0000","last_fullsized":"2024-09-11T22:12:13.699093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:55.887697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406408+0000","last_change":"2024-09-11T22:12:02.757412+0000","last_active":"2024-09-11T22:12:13.406408+0000","last_peered":"2024-09-11T22:12:13.406408+0000","last_clean":"2024-09-11T22:12:13.406408+0000","last_became_active":"2024-09-11T22:12:02.757013+0000","last_became_peered":"2024-09-11T22:12:02.757013+0000","last_unstale":"2024-09-11T22:12:13.406408+0000","last_undegraded":"2024-09-11T22:12:13.406408+0000","last_fullsized":"2024-09-11T22:12:13.406408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:21.468082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.206836+0000","last_change":"2024-09-11T22:12:02.765943+0000","last_active":"2024-09-11T22:12:13.206836+0000","last_peered":"2024-09-11T22:12:13.206836+0000","last_clean":"2024-09-11T22:12:13.206836+0000","last_became_active":"2024-09-11T22:12:02.765617+0000","last_became_peered":"2024-09-11T22:12:02.765617+0000","last_unstale":"2024-09-11T22:12:13.206836+0000","last_undegraded":"2024-09-11T22:12:13.206836+0000","last_fullsized":"2024-09-11T22:12:13.206836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:47.023509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699544+0000","last_change":"2024-09-11T22:12:02.772379+0000","last_active":"2024-09-11T22:12:13.699544+0000","last_peered":"2024-09-11T22:12:13.699544+0000","last_clean":"2024-09-11T22:12:13.699544+0000","last_became_active":"2024-09-11T22:12:02.771561+0000","last_became_peered":"2024-09-11T22:12:02.771561+0000","last_unstale":"2024-09-11T22:12:13.699544+0000","last_undegraded":"2024-09-11T22:12:13.699544+0000","last_fullsized":"2024-09-11T22:12:13.699544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:52.988744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405677+0000","last_change":"2024-09-11T22:12:02.763489+0000","last_active":"2024-09-11T22:12:13.405677+0000","last_peered":"2024-09-11T22:12:13.405677+0000","last_clean":"2024-09-11T22:12:13.405677+0000","last_became_active":"2024-09-11T22:12:02.763194+0000","last_became_peered":"2024-09-11T22:12:02.763194+0000","last_unstale":"2024-09-11T22:12:13.405677+0000","last_undegraded":"2024-09-11T22:12:13.405677+0000","last_fullsized":"2024-09-11T22:12:13.405677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:15:30.685816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405436+0000","last_change":"2024-09-11T22:12:02.768537+0000","last_active":"2024-09-11T22:12:13.405436+0000","last_peered":"2024-09-11T22:12:13.405436+0000","last_clean":"2024-09-11T22:12:13.405436+0000","last_became_active":"2024-09-11T22:12:02.768219+0000","last_became_peered":"2024-09-11T22:12:02.768219+0000","last_unstale":"2024-09-11T22:12:13.405436+0000","last_undegraded":"2024-09-11T22:12:13.405436+0000","last_fullsized":"2024-09-11T22:12:13.405436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:45:03.177587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405569+0000","last_change":"2024-09-11T22:12:02.761823+0000","last_active":"2024-09-11T22:12:13.405569+0000","last_peered":"2024-09-11T22:12:13.405569+0000","last_clean":"2024-09-11T22:12:13.405569+0000","last_became_active":"2024-09-11T22:12:02.761435+0000","last_became_peered":"2024-09-11T22:12:02.761435+0000","last_unstale":"2024-09-11T22:12:13.405569+0000","last_undegraded":"2024-09-11T22:12:13.405569+0000","last_fullsized":"2024-09-11T22:12:13.405569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:54.535178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.813827+0000","last_change":"2024-09-11T22:12:02.775613+0000","last_active":"2024-09-11T22:12:12.813827+0000","last_peered":"2024-09-11T22:12:12.813827+0000","last_clean":"2024-09-11T22:12:12.813827+0000","last_became_active":"2024-09-11T22:12:02.775383+0000","last_became_peered":"2024-09-11T22:12:02.775383+0000","last_unstale":"2024-09-11T22:12:12.813827+0000","last_undegraded":"2024-09-11T22:12:12.813827+0000","last_fullsized":"2024-09-11T22:12:12.813827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:18.973047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.407044+0000","last_change":"2024-09-11T22:12:02.771155+0000","last_active":"2024-09-11T22:12:13.407044+0000","last_peered":"2024-09-11T22:12:13.407044+0000","last_clean":"2024-09-11T22:12:13.407044+0000","last_became_active":"2024-09-11T22:12:02.770278+0000","last_became_peered":"2024-09-11T22:12:02.770278+0000","last_unstale":"2024-09-11T22:12:13.407044+0000","last_undegraded":"2024-09-11T22:12:13.407044+0000","last_fullsized":"2024-09-11T22:12:13.407044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:23.888372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814524+0000","last_change":"2024-09-11T22:12:02.772840+0000","last_active":"2024-09-11T22:12:12.814524+0000","last_peered":"2024-09-11T22:12:12.814524+0000","last_clean":"2024-09-11T22:12:12.814524+0000","last_became_active":"2024-09-11T22:12:02.772384+0000","last_became_peered":"2024-09-11T22:12:02.772384+0000","last_unstale":"2024-09-11T22:12:12.814524+0000","last_undegraded":"2024-09-11T22:12:12.814524+0000","last_fullsized":"2024-09-11T22:12:12.814524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:42.287294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.407568+0000","last_change":"2024-09-11T22:12:02.767741+0000","last_active":"2024-09-11T22:12:13.407568+0000","last_peered":"2024-09-11T22:12:13.407568+0000","last_clean":"2024-09-11T22:12:13.407568+0000","last_became_active":"2024-09-11T22:12:02.767234+0000","last_became_peered":"2024-09-11T22:12:02.767234+0000","last_unstale":"2024-09-11T22:12:13.407568+0000","last_undegraded":"2024-09-11T22:12:13.407568+0000","last_fullsized":"2024-09-11T22:12:13.407568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:22.250763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406966+0000","last_change":"2024-09-11T22:12:02.771265+0000","last_active":"2024-09-11T22:12:13.406966+0000","last_peered":"2024-09-11T22:12:13.406966+0000","last_clean":"2024-09-11T22:12:13.406966+0000","last_became_active":"2024-09-11T22:12:02.770617+0000","last_became_peered":"2024-09-11T22:12:02.770617+0000","last_unstale":"2024-09-11T22:12:13.406966+0000","last_undegraded":"2024-09-11T22:12:13.406966+0000","last_fullsized":"2024-09-11T22:12:13.406966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.390581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405722+0000","last_change":"2024-09-11T22:12:02.770850+0000","last_active":"2024-09-11T22:12:13.405722+0000","last_peered":"2024-09-11T22:12:13.405722+0000","last_clean":"2024-09-11T22:12:13.405722+0000","last_became_active":"2024-09-11T22:12:02.770559+0000","last_became_peered":"2024-09-11T22:12:02.770559+0000","last_unstale":"2024-09-11T22:12:13.405722+0000","last_undegraded":"2024-09-11T22:12:13.405722+0000","last_fullsized":"2024-09-11T22:12:13.405722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:06.865962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:12.814107+0000","last_change":"2024-09-11T22:12:02.775947+0000","last_active":"2024-09-11T22:12:12.814107+0000","last_peered":"2024-09-11T22:12:12.814107+0000","last_clean":"2024-09-11T22:12:12.814107+0000","last_became_active":"2024-09-11T22:12:02.775559+0000","last_became_peered":"2024-09-11T22:12:02.775559+0000","last_unstale":"2024-09-11T22:12:12.814107+0000","last_undegraded":"2024-09-11T22:12:12.814107+0000","last_fullsized":"2024-09-11T22:12:12.814107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:14.642600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406122+0000","last_change":"2024-09-11T22:12:02.768124+0000","last_active":"2024-09-11T22:12:13.406122+0000","last_peered":"2024-09-11T22:12:13.406122+0000","last_clean":"2024-09-11T22:12:13.406122+0000","last_became_active":"2024-09-11T22:12:02.767804+0000","last_became_peered":"2024-09-11T22:12:02.767804+0000","last_unstale":"2024-09-11T22:12:13.406122+0000","last_undegraded":"2024-09-11T22:12:13.406122+0000","last_fullsized":"2024-09-11T22:12:13.406122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:23.612445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406414+0000","last_change":"2024-09-11T22:12:02.764086+0000","last_active":"2024-09-11T22:12:13.406414+0000","last_peered":"2024-09-11T22:12:13.406414+0000","last_clean":"2024-09-11T22:12:13.406414+0000","last_became_active":"2024-09-11T22:12:02.763708+0000","last_became_peered":"2024-09-11T22:12:02.763708+0000","last_unstale":"2024-09-11T22:12:13.406414+0000","last_undegraded":"2024-09-11T22:12:13.406414+0000","last_fullsized":"2024-09-11T22:12:13.406414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:06.058314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.406319+0000","last_change":"2024-09-11T22:12:02.770820+0000","last_active":"2024-09-11T22:12:13.406319+0000","last_peered":"2024-09-11T22:12:13.406319+0000","last_clean":"2024-09-11T22:12:13.406319+0000","last_became_active":"2024-09-11T22:12:02.767911+0000","last_became_peered":"2024-09-11T22:12:02.767911+0000","last_unstale":"2024-09-11T22:12:13.406319+0000","last_undegraded":"2024-09-11T22:12:13.406319+0000","last_fullsized":"2024-09-11T22:12:13.406319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:12:04.030699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405899+0000","last_change":"2024-09-11T22:12:02.764702+0000","last_active":"2024-09-11T22:12:13.405899+0000","last_peered":"2024-09-11T22:12:13.405899+0000","last_clean":"2024-09-11T22:12:13.405899+0000","last_became_active":"2024-09-11T22:12:02.764407+0000","last_became_peered":"2024-09-11T22:12:02.764407+0000","last_unstale":"2024-09-11T22:12:13.405899+0000","last_undegraded":"2024-09-11T22:12:13.405899+0000","last_fullsized":"2024-09-11T22:12:13.405899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:01.551823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.405722+0000","last_change":"2024-09-11T22:12:02.762209+0000","last_active":"2024-09-11T22:12:13.405722+0000","last_peered":"2024-09-11T22:12:13.405722+0000","last_clean":"2024-09-11T22:12:13.405722+0000","last_became_active":"2024-09-11T22:12:02.761936+0000","last_became_peered":"2024-09-11T22:12:02.761936+0000","last_unstale":"2024-09-11T22:12:13.405722+0000","last_undegraded":"2024-09-11T22:12:13.405722+0000","last_fullsized":"2024-09-11T22:12:13.405722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:39.667680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-11T22:12:13.699536+0000","last_change":"2024-09-11T22:12:02.773537+0000","last_active":"2024-09-11T22:12:13.699536+0000","last_peered":"2024-09-11T22:12:13.699536+0000","last_clean":"2024-09-11T22:12:13.699536+0000","last_became_active":"2024-09-11T22:12:02.773145+0000","last_became_peered":"2024-09-11T22:12:02.773145+0000","last_unstale":"2024-09-11T22:12:13.699536+0000","last_undegraded":"2024-09-11T22:12:13.699536+0000","last_fullsized":"2024-09-11T22:12:13.699536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:30.058536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56306,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":151,"num_read_kb":131,"num_write":233,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":176569,"data_compressed":5631,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286124,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449109,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":491369,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1127676,"data_compressed":55903,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":495395,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":491677,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299435,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":1088569,"data_compressed":54026,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462958,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":487196,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691892,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1083365,"data_compressed":54026,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855418,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":491270,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117055,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":486511,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":544802,"data_compressed":19524,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":537324,"data_compressed":19524,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":488259,"data_compressed":17647,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":51243,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":58721,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":51227,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:12:55.778 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:12:55.779 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T22:12:55.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T22:12:56.023 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 -- 172.21.15.92:0/970499143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f01037c0 msgr2=0x7fe6f010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 --2- 172.21.15.92:0/970499143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f01037c0 0x7fe6f010a050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009ca0 tx=0x7fe6e802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 -- 172.21.15.92:0/970499143 shutdown_connections 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 --2- 172.21.15.92:0/970499143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f01037c0 0x7fe6f010a050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 --2- 172.21.15.92:0/970499143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6f0102e00 0x7fe6f0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 --2- 172.21.15.92:0/970499143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f0101c00 0x7fe6f0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 -- 172.21.15.92:0/970499143 >> 172.21.15.92:0/970499143 conn(0x7fe6f00fd390 msgr2=0x7fe6f00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 -- 172.21.15.92:0/970499143 shutdown_connections 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 -- 172.21.15.92:0/970499143 wait complete. 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 Processor -- start 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.514+0000 7fe6f4952640 1 -- start start 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6f4952640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f0101c00 0x7fe6f0197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6f4952640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6f0102e00 0x7fe6f01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6f4952640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f01037c0 0x7fe6f019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6f4952640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f010d710 con 0x7fe6f0101c00 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6f4952640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f010d590 con 0x7fe6f01037c0 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6f4952640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f010d890 con 0x7fe6f0102e00 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f0101c00 0x7fe6f0197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f0101c00 0x7fe6f0197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40114/0 (socket says 172.21.15.92:40114) 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 -- 172.21.15.92:0/2845115363 learned_addr learned my addr 172.21.15.92:0/2845115363 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6eed9d640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f01037c0 0x7fe6f019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6edd9b640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6f0102e00 0x7fe6f01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 -- 172.21.15.92:0/2845115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6f0102e00 msgr2=0x7fe6f01981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6f0102e00 0x7fe6f01981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 -- 172.21.15.92:0/2845115363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f01037c0 msgr2=0x7fe6f019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f01037c0 0x7fe6f019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 -- 172.21.15.92:0/2845115363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f019f8d0 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6edd9b640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6f0102e00 0x7fe6f01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6ee59c640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f0101c00 0x7fe6f0197cb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000c970 tx=0x7fe6e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6df7fe640 1 -- 172.21.15.92:0/2845115363 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0007c10 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6df7fe640 1 -- 172.21.15.92:0/2845115363 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6e0007db0 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6df7fe640 1 -- 172.21.15.92:0/2845115363 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000f870 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.515+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f019fbc0 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.516+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f01a0100 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.517+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f0108fb0 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.518+0000 7fe6df7fe640 1 -- 172.21.15.92:0/2845115363 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe6e0043b10 con 0x7fe6f0101c00 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.518+0000 7fe6df7fe640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe6d80a2c10 0x7fe6d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.518+0000 7fe6edd9b640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe6d80a2c10 0x7fe6d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:56.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.518+0000 7fe6df7fe640 1 -- 172.21.15.92:0/2845115363 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe6e00cf8d0 con 0x7fe6f0101c00 2024-09-11T22:12:56.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.519+0000 7fe6edd9b640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe6d80a2c10 0x7fe6d80a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe6f01991c0 tx=0x7fe6e400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:56.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.522+0000 7fe6df7fe640 1 -- 172.21.15.92:0/2845115363 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6e009c4c0 con 0x7fe6f0101c00 2024-09-11T22:12:56.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.752+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe6f002d7c0 con 0x7fe6f0101c00 2024-09-11T22:12:56.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.754+0000 7fe6df7fe640 1 -- 172.21.15.92:0/2845115363 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe6e009bdc0 con 0x7fe6f0101c00 2024-09-11T22:12:56.755 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-09-11T22:12:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDoFeJmHm/3LBAA5qBPomnXZKSu9gCgplcH/Q== 2024-09-11T22:12:56.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe6d80a2c10 msgr2=0x7fe6d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe6d80a2c10 0x7fe6d80a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe6f01991c0 tx=0x7fe6e400a5c0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f0101c00 msgr2=0x7fe6f0197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f0101c00 0x7fe6f0197cb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000c970 tx=0x7fe6e000ce30 comp rx=0 tx=0).stop 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 shutdown_connections 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fe6d80a2c10 0x7fe6d80a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f01037c0 0x7fe6f019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6f0102e00 0x7fe6f01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 --2- 172.21.15.92:0/2845115363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6f0101c00 0x7fe6f0197cb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 >> 172.21.15.92:0/2845115363 conn(0x7fe6f00fd390 msgr2=0x7fe6f0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.756+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 shutdown_connections 2024-09-11T22:12:56.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:56.757+0000 7fe6f4952640 1 -- 172.21.15.92:0/2845115363 wait complete. 2024-09-11T22:12:57.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:56 smithi092 ceph-mon[31173]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:12:57.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:56 smithi092 ceph-mon[31173]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-09-11T22:12:57.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:56 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2845115363' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:12:57.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:56 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2845115363' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:12:57.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:56 smithi153 ceph-mon[35045]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:12:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:56 smithi153 ceph-mon[35045]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-09-11T22:12:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:56 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2845115363' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:12:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:56 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2845115363' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:12:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:56 smithi110 ceph-mon[34830]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:12:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:56 smithi110 ceph-mon[34830]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-09-11T22:12:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:56 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2845115363' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:12:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:56 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2845115363' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:12:57.383 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:12:57.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T22:12:57.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T22:12:57.420 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T22:12:57.420 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T22:12:57.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T22:12:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 -- 172.21.15.92:0/132191688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec101df0 msgr2=0x7f36ec1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/132191688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec101df0 0x7f36ec1021d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009ca0 tx=0x7f36dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 -- 172.21.15.92:0/132191688 shutdown_connections 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/132191688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec106f20 0x7f36ec10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/132191688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36ec1027a0 0x7f36ec1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/132191688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec101df0 0x7f36ec1021d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 -- 172.21.15.92:0/132191688 >> 172.21.15.92:0/132191688 conn(0x7f36ec0fd8f0 msgr2=0x7f36ec0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 -- 172.21.15.92:0/132191688 shutdown_connections 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 -- 172.21.15.92:0/132191688 wait complete. 2024-09-11T22:12:58.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 Processor -- start 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.183+0000 7f36f3fbc640 1 -- start start 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec101df0 0x7f36ec1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec1027a0 0x7f36ec1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36ec106f20 0x7f36ec19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111e80 con 0x7f36ec1027a0 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111d00 con 0x7f36ec106f20 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec112000 con 0x7f36ec101df0 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec1027a0 0x7f36ec1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec1027a0 0x7f36ec1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40126/0 (socket says 172.21.15.92:40126) 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 -- 172.21.15.92:0/530729078 learned_addr learned my addr 172.21.15.92:0/530729078 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1d58640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec101df0 0x7f36ec1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f2559640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36ec106f20 0x7f36ec19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 -- 172.21.15.92:0/530729078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec101df0 msgr2=0x7f36ec1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec101df0 0x7f36ec1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 -- 172.21.15.92:0/530729078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36ec106f20 msgr2=0x7f36ec19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36ec106f20 0x7f36ec19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 -- 172.21.15.92:0/530729078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec19d350 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f1557640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec1027a0 0x7f36ec1988e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d950 tx=0x7f36e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36eaffd640 1 -- 172.21.15.92:0/530729078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0017030 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36eaffd640 1 -- 172.21.15.92:0/530729078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e0004020 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec19d640 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36eaffd640 1 -- 172.21.15.92:0/530729078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00047a0 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.184+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a4f20 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.185+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ec1021d0 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.186+0000 7f36eaffd640 1 -- 172.21.15.92:0/530729078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f36e00041d0 con 0x7f36ec1027a0 2024-09-11T22:12:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.187+0000 7f36eaffd640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f36d40a2c90 0x7f36d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.187+0000 7f36f1d58640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f36d40a2c90 0x7f36d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.187+0000 7f36eaffd640 1 -- 172.21.15.92:0/530729078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f36e00ced70 con 0x7f36ec1027a0 2024-09-11T22:12:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.188+0000 7f36f1d58640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f36d40a2c90 0x7f36d40a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36dc006010 tx=0x7f36dc031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.191+0000 7f36eaffd640 1 -- 172.21.15.92:0/530729078 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36e009b930 con 0x7f36ec1027a0 2024-09-11T22:12:58.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.425+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f36ec10a130 con 0x7f36ec1027a0 2024-09-11T22:12:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.430+0000 7f36eaffd640 1 -- 172.21.15.92:0/530729078 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7f36e009b230 con 0x7f36ec1027a0 2024-09-11T22:12:58.433 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:12:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.432+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f36d40a2c90 msgr2=0x7f36d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.432+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f36d40a2c90 0x7f36d40a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36dc006010 tx=0x7f36dc031000 comp rx=0 tx=0).stop 2024-09-11T22:12:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.432+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec1027a0 msgr2=0x7f36ec1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.432+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec1027a0 0x7f36ec1988e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d950 tx=0x7f36e000de10 comp rx=0 tx=0).stop 2024-09-11T22:12:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 shutdown_connections 2024-09-11T22:12:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f36d40a2c90 0x7f36d40a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36ec106f20 0x7f36ec19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec1027a0 0x7f36ec1988e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 --2- 172.21.15.92:0/530729078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec101df0 0x7f36ec1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:58.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 >> 172.21.15.92:0/530729078 conn(0x7f36ec0fd8f0 msgr2=0x7f36ec0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:58.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 shutdown_connections 2024-09-11T22:12:58.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:12:58.433+0000 7f36f3fbc640 1 -- 172.21.15.92:0/530729078 wait complete. 2024-09-11T22:12:59.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:58 smithi092 ceph-mon[31173]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-11T22:12:59.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:58 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:12:59.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:12:58 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/530729078' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:12:59.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:58 smithi153 ceph-mon[35045]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-11T22:12:59.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:58 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:12:59.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:12:58 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/530729078' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:12:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:58 smithi110 ceph-mon[34830]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-11T22:12:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:58 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:12:59.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:12:58 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/530729078' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:12:59.674 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1619835932},{"type":"v1","addr":"172.21.15.92:6801","nonce":1619835932}]},"active_addr":"172.21.15.92:6801/1619835932","active_change":"2024-09-11T22:05:15.603553+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.92:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":4100905256}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2742597002}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3586351080}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1517821039}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":302544560}]}]} 2024-09-11T22:12:59.680 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T22:12:59.681 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T22:12:59.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:12:59.959 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:00.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 -- 172.21.15.92:0/3397586689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c100e20 msgr2=0x7f673c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:00.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3397586689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c100e20 0x7f673c101200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f672c009900 tx=0x7f672c02a480 comp rx=0 tx=0).stop 2024-09-11T22:13:00.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 -- 172.21.15.92:0/3397586689 shutdown_connections 2024-09-11T22:13:00.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3397586689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c102ef0 0x7f673c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3397586689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f673c102570 0x7f673c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3397586689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c100e20 0x7f673c101200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 -- 172.21.15.92:0/3397586689 >> 172.21.15.92:0/3397586689 conn(0x7f673c0fc7d0 msgr2=0x7f673c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 -- 172.21.15.92:0/3397586689 shutdown_connections 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.466+0000 7f6742a0f640 1 -- 172.21.15.92:0/3397586689 wait complete. 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 Processor -- start 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 -- start start 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f673c100e20 0x7f673c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c102570 0x7f673c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c102ef0 0x7f673c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f673c110ea0 con 0x7f673c102570 2024-09-11T22:13:00.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f673c110d20 con 0x7f673c100e20 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f6742a0f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111020 con 0x7f673c102ef0 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c102570 0x7f673c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c102570 0x7f673c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40140/0 (socket says 172.21.15.92:40140) 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.467+0000 7f673b7fe640 1 -- 172.21.15.92:0/3828015406 learned_addr learned my addr 172.21.15.92:0/3828015406 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673bfff640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f673c100e20 0x7f673c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f6740fac640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c102ef0 0x7f673c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673b7fe640 1 -- 172.21.15.92:0/3828015406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c102ef0 msgr2=0x7f673c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673b7fe640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c102ef0 0x7f673c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673b7fe640 1 -- 172.21.15.92:0/3828015406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f673c100e20 msgr2=0x7f673c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673b7fe640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f673c100e20 0x7f673c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673b7fe640 1 -- 172.21.15.92:0/3828015406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c19c3d0 con 0x7f673c102570 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673bfff640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f673c100e20 0x7f673c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f673b7fe640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c102570 0x7f673c197960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f673000b4a0 tx=0x7f673000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f67397fa640 1 -- 172.21.15.92:0/3828015406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730014070 con 0x7f673c102570 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c19c6c0 con 0x7f673c102570 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f67397fa640 1 -- 172.21.15.92:0/3828015406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6730010ac0 con 0x7f673c102570 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f67397fa640 1 -- 172.21.15.92:0/3828015406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730017c30 con 0x7f673c102570 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.468+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a3fa0 con 0x7f673c102570 2024-09-11T22:13:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.469+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c101200 con 0x7f673c102570 2024-09-11T22:13:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.470+0000 7f67397fa640 1 -- 172.21.15.92:0/3828015406 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6730017070 con 0x7f673c102570 2024-09-11T22:13:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.471+0000 7f67397fa640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f67240a2c10 0x7f67240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.471+0000 7f67397fa640 1 -- 172.21.15.92:0/3828015406 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f67300ceb30 con 0x7f673c102570 2024-09-11T22:13:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.471+0000 7f673bfff640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f67240a2c10 0x7f67240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.471+0000 7f673bfff640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f67240a2c10 0x7f67240a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f672c000c00 tx=0x7f672c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.474+0000 7f67397fa640 1 -- 172.21.15.92:0/3828015406 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f673009b800 con 0x7f673c102570 2024-09-11T22:13:00.653 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.654 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.654 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.654 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:00 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.654 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:00 smithi092 ceph-mon[31173]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-11T22:13:00.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.653+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f673c198780 con 0x7f673c102570 2024-09-11T22:13:00.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.655+0000 7f67397fa640 1 -- 172.21.15.92:0/3828015406 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7f673c198780 con 0x7f673c102570 2024-09-11T22:13:00.655 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:00.656 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f67240a2c10 msgr2=0x7f67240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f67240a2c10 0x7f67240a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f672c000c00 tx=0x7f672c005a20 comp rx=0 tx=0).stop 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c102570 msgr2=0x7f673c197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c102570 0x7f673c197960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f673000b4a0 tx=0x7f673000b960 comp rx=0 tx=0).stop 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 shutdown_connections 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f67240a2c10 0x7f67240a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c102ef0 0x7f673c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f673c102570 0x7f673c197960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 --2- 172.21.15.92:0/3828015406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f673c100e20 0x7f673c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:00.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 >> 172.21.15.92:0/3828015406 conn(0x7f673c0fc7d0 msgr2=0x7f673c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:00.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 shutdown_connections 2024-09-11T22:13:00.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:00.659+0000 7f6742a0f640 1 -- 172.21.15.92:0/3828015406 wait complete. 2024-09-11T22:13:00.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:00 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:00 smithi110 ceph-mon[34830]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-11T22:13:00.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:00 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:00.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:00 smithi153 ceph-mon[35045]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-11T22:13:01.393 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T22:13:01.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:13:01.639 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:01.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:01 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3828015406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:01.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:01.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:13:01.760 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:01 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3828015406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:01.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:01.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:13:01.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:01.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:01 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3828015406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:01.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:01.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:13:01.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:13:02.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.083+0000 7f9bebc63640 1 -- 172.21.15.92:0/3631392263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 msgr2=0x7f9be41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:02.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.083+0000 7f9bebc63640 1 --2- 172.21.15.92:0/3631392263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be41021c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009900 tx=0x7f9bd402a410 comp rx=0 tx=0).stop 2024-09-11T22:13:02.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.083+0000 7f9bebc63640 1 -- 172.21.15.92:0/3631392263 shutdown_connections 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.083+0000 7f9bebc63640 1 --2- 172.21.15.92:0/3631392263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be4106f10 0x7f9be410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 --2- 172.21.15.92:0/3631392263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be4102790 0x7f9be41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 --2- 172.21.15.92:0/3631392263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be41021c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 -- 172.21.15.92:0/3631392263 >> 172.21.15.92:0/3631392263 conn(0x7f9be40fd8c0 msgr2=0x7f9be40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 -- 172.21.15.92:0/3631392263 shutdown_connections 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 -- 172.21.15.92:0/3631392263 wait complete. 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 Processor -- start 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 -- start start 2024-09-11T22:13:02.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be4102790 0x7f9be4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be4106f10 0x7f9be4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9be4111e20 con 0x7f9be4101de0 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9be4111ca0 con 0x7f9be4106f10 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.084+0000 7f9bebc63640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9be4111fa0 con 0x7f9be4102790 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40160/0 (socket says 172.21.15.92:40160) 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 -- 172.21.15.92:0/1313032897 learned_addr learned my addr 172.21.15.92:0/1313032897 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9bea200640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be4106f10 0x7f9be4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be91fe640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be4102790 0x7f9be4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 -- 172.21.15.92:0/1313032897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be4102790 msgr2=0x7f9be4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be4102790 0x7f9be4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 -- 172.21.15.92:0/1313032897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be4106f10 msgr2=0x7f9be4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be4106f10 0x7f9be4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 -- 172.21.15.92:0/1313032897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be41a6820 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9bea200640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be4106f10 0x7f9be4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be91fe640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be4102790 0x7f9be4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9be99ff640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be40744f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005940 tx=0x7f9bd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9bdaffd640 1 -- 172.21.15.92:0/1313032897 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd403f030 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9bdaffd640 1 -- 172.21.15.92:0/1313032897 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bd4003aa0 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be41a6ab0 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9bdaffd640 1 -- 172.21.15.92:0/1313032897 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4035070 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.085+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be41a6fe0 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.087+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be41021c0 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.087+0000 7f9bdaffd640 1 -- 172.21.15.92:0/1313032897 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9bd4042030 con 0x7f9be4101de0 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.088+0000 7f9bdaffd640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9bd00a2c90 0x7f9bd00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:02.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.088+0000 7f9be91fe640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9bd00a2c90 0x7f9bd00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:02.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.089+0000 7f9bdaffd640 1 -- 172.21.15.92:0/1313032897 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9bd40efc10 con 0x7f9be4101de0 2024-09-11T22:13:02.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.089+0000 7f9be91fe640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9bd00a2c90 0x7f9bd00a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9be4075f80 tx=0x7f9be0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:02.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.093+0000 7f9bdaffd640 1 -- 172.21.15.92:0/1313032897 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bd40bc7d0 con 0x7f9be4101de0 2024-09-11T22:13:02.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.261+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9be4109b60 con 0x7f9be4101de0 2024-09-11T22:13:02.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.263+0000 7f9bdaffd640 1 -- 172.21.15.92:0/1313032897 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24211 (secure 0 0 0) 0x7f9bd40bc0d0 con 0x7f9be4101de0 2024-09-11T22:13:02.263 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:02.263 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:12:12.449315+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:13:02.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9bd00a2c90 msgr2=0x7f9bd00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:02.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9bd00a2c90 0x7f9bd00a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9be4075f80 tx=0x7f9be0005e50 comp rx=0 tx=0).stop 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 msgr2=0x7f9be40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be40744f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005940 tx=0x7f9bd4004270 comp rx=0 tx=0).stop 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 shutdown_connections 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f9bd00a2c90 0x7f9bd00a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be4106f10 0x7f9be4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be4102790 0x7f9be4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 --2- 172.21.15.92:0/1313032897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be4101de0 0x7f9be40744f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 >> 172.21.15.92:0/1313032897 conn(0x7f9be40fd8c0 msgr2=0x7f9be40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 shutdown_connections 2024-09-11T22:13:02.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:02.265+0000 7f9bebc63640 1 -- 172.21.15.92:0/1313032897 wait complete. 2024-09-11T22:13:02.908 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:02 smithi092 ceph-mon[31173]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-11T22:13:02.908 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:02 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1313032897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:02.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T22:13:02.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T22:13:02.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T22:13:02.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T22:13:02.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T22:13:02.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T22:13:02.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T22:13:02.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T22:13:02.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T22:13:02.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T22:13:02.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T22:13:02.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T22:13:02.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:02 smithi110 ceph-mon[34830]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-11T22:13:02.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:02 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1313032897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:02 smithi153 ceph-mon[35045]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-11T22:13:03.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:02 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1313032897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:03.664 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:03.700 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:03.709 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:03.710 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:03.712 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:03.713 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:03.713 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:04.877 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:04 smithi092 ceph-mon[31173]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:13:05.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 -- 172.21.15.92:0/4152205636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f881010c190 msgr2=0x7f881010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 --2- 172.21.15.92:0/4152205636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f881010c190 0x7f881010e610 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802a480 comp rx=0 tx=0).stop 2024-09-11T22:13:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 -- 172.21.15.92:0/4152205636 shutdown_connections 2024-09-11T22:13:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 --2- 172.21.15.92:0/4152205636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f881010c190 0x7f881010e610 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 --2- 172.21.15.92:0/4152205636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88101036d0 0x7f881010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 --2- 172.21.15.92:0/4152205636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8810102db0 0x7f8810103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 -- 172.21.15.92:0/4152205636 >> 172.21.15.92:0/4152205636 conn(0x7f88100fc7d0 msgr2=0x7f88100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.012+0000 7f8815478640 1 -- 172.21.15.92:0/4152205636 shutdown_connections 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 -- 172.21.15.92:0/4152205636 wait complete. 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 Processor -- start 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 -- start start 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8810102db0 0x7f88101983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88101036d0 0x7f88101988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f881010c190 0x7f881019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8810111e80 con 0x7f8810102db0 2024-09-11T22:13:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8810111d00 con 0x7f881010c190 2024-09-11T22:13:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.013+0000 7f8815478640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8810112000 con 0x7f88101036d0 2024-09-11T22:13:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8810102db0 0x7f88101983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8810102db0 0x7f88101983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40178/0 (socket says 172.21.15.92:40178) 2024-09-11T22:13:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 -- 172.21.15.92:0/783767716 learned_addr learned my addr 172.21.15.92:0/783767716 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880f7fe640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f881010c190 0x7f881019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880e7fc640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88101036d0 0x7f88101988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 -- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88101036d0 msgr2=0x7f88101988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88101036d0 0x7f88101988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 -- 172.21.15.92:0/783767716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f881010c190 msgr2=0x7f881019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f881010c190 0x7f881019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 -- 172.21.15.92:0/783767716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881019d360 con 0x7f8810102db0 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880e7fc640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88101036d0 0x7f88101988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f880effd640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8810102db0 0x7f88101983b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f880000ba50 tx=0x7f880000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800019070 con 0x7f8810102db0 2024-09-11T22:13:05.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f880000cbe0 con 0x7f8810102db0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881019d650 con 0x7f8810102db0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000075d0 con 0x7f8810102db0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.014+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88101a4f30 con 0x7f8810102db0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.015+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8810062900 con 0x7f8810102db0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.017+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f880001c070 con 0x7f8810102db0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.017+0000 7f8807fff640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87fc0a2c10 0x7f87fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.017+0000 7f880e7fc640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87fc0a2c10 0x7f87fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.017+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f88000cf130 con 0x7f8810102db0 2024-09-11T22:13:05.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.018+0000 7f8807fff640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] conn(0x7f87fc0ad940 0x7f87fc0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.018+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 --> [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8800016ec0 con 0x7f87fc0ad940 2024-09-11T22:13:05.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.018+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f88000cf580 con 0x7f8810102db0 2024-09-11T22:13:05.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.018+0000 7f880e7fc640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87fc0a2c10 0x7f87fc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88100fee40 tx=0x7f87f8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.018+0000 7f880f7fe640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] conn(0x7f87fc0ad940 0x7f87fc0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.018+0000 7f880f7fe640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] conn(0x7f87fc0ad940 0x7f87fc0afd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.019+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== osd.8 v2:172.21.15.153:6816/979097601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8800016ec0 con 0x7f87fc0ad940 2024-09-11T22:13:05.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.033+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 --> [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8810062d80 con 0x7f87fc0ad940 2024-09-11T22:13:05.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.033+0000 7f8807fff640 1 -- 172.21.15.92:0/783767716 <== osd.8 v2:172.21.15.153:6816/979097601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8810062d80 con 0x7f87fc0ad940 2024-09-11T22:13:05.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] conn(0x7f87fc0ad940 msgr2=0x7f87fc0afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] conn(0x7f87fc0ad940 0x7f87fc0afd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87fc0a2c10 msgr2=0x7f87fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f87fc0a2c10 0x7f87fc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88100fee40 tx=0x7f87f8005d30 comp rx=0 tx=0).stop 2024-09-11T22:13:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8810102db0 msgr2=0x7f88101983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 --2- 172.21.15.92:0/783767716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8810102db0 0x7f88101983b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f880000ba50 tx=0x7f880000bf10 comp rx=0 tx=0).stop 2024-09-11T22:13:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f880f7fe640 1 -- 172.21.15.92:0/783767716 reap_dead start 2024-09-11T22:13:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 shutdown_connections 2024-09-11T22:13:05.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 >> 172.21.15.92:0/783767716 conn(0x7f88100fc7d0 msgr2=0x7f88100fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:05.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 shutdown_connections 2024-09-11T22:13:05.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.034+0000 7f8815478640 1 -- 172.21.15.92:0/783767716 wait complete. 2024-09-11T22:13:05.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:04 smithi153 ceph-mon[35045]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:13:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:04 smithi110 ceph-mon[34830]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:13:05.743 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:05.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 -- 172.21.15.92:0/3863244817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c101420 msgr2=0x7ff23c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 --2- 172.21.15.92:0/3863244817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c101420 0x7ff23c10d950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009cd0 tx=0x7ff22c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 -- 172.21.15.92:0/3863244817 shutdown_connections 2024-09-11T22:13:05.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 --2- 172.21.15.92:0/3863244817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c101420 0x7ff23c10d950 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 --2- 172.21.15.92:0/3863244817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff23c100a80 0x7ff23c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 --2- 172.21.15.92:0/3863244817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c106a80 0x7ff23c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 -- 172.21.15.92:0/3863244817 >> 172.21.15.92:0/3863244817 conn(0x7ff23c0fc7a0 msgr2=0x7ff23c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 -- 172.21.15.92:0/3863244817 shutdown_connections 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 -- 172.21.15.92:0/3863244817 wait complete. 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.593+0000 7ff2445d5640 1 Processor -- start 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff2445d5640 1 -- start start 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff2445d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c100a80 0x7ff23c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff2445d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff23c101420 0x7ff23c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff2445d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c106a80 0x7ff23c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff2445d5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c110d30 con 0x7ff23c106a80 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff2445d5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c110bb0 con 0x7ff23c100a80 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff2445d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c110eb0 con 0x7ff23c101420 2024-09-11T22:13:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c106a80 0x7ff23c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c106a80 0x7ff23c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51460/0 (socket says 172.21.15.92:51460) 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 -- 172.21.15.92:0/3008111810 learned_addr learned my addr 172.21.15.92:0/3008111810 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff241b70640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff23c101420 0x7ff23c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242371640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c100a80 0x7ff23c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 -- 172.21.15.92:0/3008111810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff23c101420 msgr2=0x7ff23c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff23c101420 0x7ff23c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 -- 172.21.15.92:0/3008111810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c100a80 msgr2=0x7ff23c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c100a80 0x7ff23c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 -- 172.21.15.92:0/3008111810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c19d2e0 con 0x7ff23c106a80 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.594+0000 7ff242b72640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c106a80 0x7ff23c19cc00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff22c005cc0 tx=0x7ff22c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.595+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c004450 con 0x7ff23c106a80 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.595+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c19d570 con 0x7ff23c106a80 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.595+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff22c031c90 con 0x7ff23c106a80 2024-09-11T22:13:05.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.595+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c03b400 con 0x7ff23c106a80 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.595+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1a4eb0 con 0x7ff23c106a80 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.595+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff23c062980 con 0x7ff23c106a80 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.597+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff22c004680 con 0x7ff23c106a80 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.597+0000 7ff23b7fe640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff2240a2c10 0x7ff2240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.597+0000 7ff242371640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff2240a2c10 0x7ff2240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.598+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff22c02e030 con 0x7ff23c106a80 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.598+0000 7ff23b7fe640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] conn(0x7ff2240ad940 0x7ff2240afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.598+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 --> [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff22c0059a0 con 0x7ff2240ad940 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.598+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff22c0f0d80 con 0x7ff23c106a80 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.598+0000 7ff241b70640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] conn(0x7ff2240ad940 0x7ff2240afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.598+0000 7ff242371640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff2240a2c10 0x7ff2240a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff234005e00 tx=0x7ff234005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.598+0000 7ff241b70640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] conn(0x7ff2240ad940 0x7ff2240afd80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.599+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== osd.0 v2:172.21.15.92:6802/2928117998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff22c0059a0 con 0x7ff2240ad940 2024-09-11T22:13:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.612+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 --> [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff23c062d80 con 0x7ff2240ad940 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.612+0000 7ff23b7fe640 1 -- 172.21.15.92:0/3008111810 <== osd.0 v2:172.21.15.92:6802/2928117998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff23c062d80 con 0x7ff2240ad940 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] conn(0x7ff2240ad940 msgr2=0x7ff2240afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] conn(0x7ff2240ad940 0x7ff2240afd80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff2240a2c10 msgr2=0x7ff2240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff2240a2c10 0x7ff2240a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff234005e00 tx=0x7ff234005c50 comp rx=0 tx=0).stop 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c106a80 msgr2=0x7ff23c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 --2- 172.21.15.92:0/3008111810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23c106a80 0x7ff23c19cc00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff22c005cc0 tx=0x7ff22c003c20 comp rx=0 tx=0).stop 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff242b72640 1 -- 172.21.15.92:0/3008111810 reap_dead start 2024-09-11T22:13:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 shutdown_connections 2024-09-11T22:13:05.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 >> 172.21.15.92:0/3008111810 conn(0x7ff23c0fc7a0 msgr2=0x7ff23c06aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:05.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 shutdown_connections 2024-09-11T22:13:05.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.613+0000 7ff2445d5640 1 -- 172.21.15.92:0/3008111810 wait complete. 2024-09-11T22:13:06.433 INFO:teuthology.orchestra.run.smithi092.stdout:34359738448 2024-09-11T22:13:06.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T22:13:06.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 -- 172.21.15.92:0/2112311057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38106a80 msgr2=0x7fda38106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 --2- 172.21.15.92:0/2112311057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38106a80 0x7fda38106e60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fda28009930 tx=0x7fda2802a420 comp rx=0 tx=0).stop 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 -- 172.21.15.92:0/2112311057 shutdown_connections 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 --2- 172.21.15.92:0/2112311057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda38101420 0x7fda3810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 --2- 172.21.15.92:0/2112311057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda38100a80 0x7fda38100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 --2- 172.21.15.92:0/2112311057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38106a80 0x7fda38106e60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 -- 172.21.15.92:0/2112311057 >> 172.21.15.92:0/2112311057 conn(0x7fda380fc7a0 msgr2=0x7fda380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 -- 172.21.15.92:0/2112311057 shutdown_connections 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.684+0000 7fda3f718640 1 -- 172.21.15.92:0/2112311057 wait complete. 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 Processor -- start 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 -- start start 2024-09-11T22:13:06.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda38100a80 0x7fda38198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38101420 0x7fda381988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda38106a80 0x7fda3819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda38110cf0 con 0x7fda38101420 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fda38110b70 con 0x7fda38100a80 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3f718640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda38110e70 con 0x7fda38106a80 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3ccb3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38101420 0x7fda381988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3ccb3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38101420 0x7fda381988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51510/0 (socket says 172.21.15.92:51510) 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3ccb3640 1 -- 172.21.15.92:0/1146327267 learned_addr learned my addr 172.21.15.92:0/1146327267 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3d4b4640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda38100a80 0x7fda38198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.685+0000 7fda3dcb5640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda38106a80 0x7fda3819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3ccb3640 1 -- 172.21.15.92:0/1146327267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda38106a80 msgr2=0x7fda3819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3ccb3640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda38106a80 0x7fda3819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3ccb3640 1 -- 172.21.15.92:0/1146327267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda38100a80 msgr2=0x7fda38198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3ccb3640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda38100a80 0x7fda38198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3ccb3640 1 -- 172.21.15.92:0/1146327267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3819d310 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3dcb5640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda38106a80 0x7fda3819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3ccb3640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38101420 0x7fda381988a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c920 tx=0x7fda3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34007c70 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda34007e10 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3819d600 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34018820 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.686+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda381a4ee0 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.687+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda38062940 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.688+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda340189c0 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda2e7fc640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda240a2c10 0x7fda240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda3d4b4640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda240a2c10 0x7fda240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fda340cfa60 con 0x7fda38101420 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda2e7fc640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] conn(0x7fda240ad940 0x7fda240afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 --> [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda34015e70 con 0x7fda240ad940 2024-09-11T22:13:06.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fda340cfeb0 con 0x7fda38101420 2024-09-11T22:13:06.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda3dcb5640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] conn(0x7fda240ad940 0x7fda240afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.689+0000 7fda3d4b4640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda240a2c10 0x7fda240a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda28032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:06.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.690+0000 7fda3dcb5640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] conn(0x7fda240ad940 0x7fda240afd80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:06.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.690+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== osd.6 v2:172.21.15.92:6818/433608050 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda34015e70 con 0x7fda240ad940 2024-09-11T22:13:06.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.707+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 --> [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda38062dc0 con 0x7fda240ad940 2024-09-11T22:13:06.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.707+0000 7fda2e7fc640 1 -- 172.21.15.92:0/1146327267 <== osd.6 v2:172.21.15.92:6818/433608050 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fda38062dc0 con 0x7fda240ad940 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] conn(0x7fda240ad940 msgr2=0x7fda240afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] conn(0x7fda240ad940 0x7fda240afd80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda240a2c10 msgr2=0x7fda240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fda240a2c10 0x7fda240a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda28032c80 comp rx=0 tx=0).stop 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38101420 msgr2=0x7fda381988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 --2- 172.21.15.92:0/1146327267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda38101420 0x7fda381988a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c920 tx=0x7fda3400cde0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3dcb5640 1 -- 172.21.15.92:0/1146327267 reap_dead start 2024-09-11T22:13:06.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 shutdown_connections 2024-09-11T22:13:06.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 >> 172.21.15.92:0/1146327267 conn(0x7fda380fc7a0 msgr2=0x7fda3806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:06.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 shutdown_connections 2024-09-11T22:13:06.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.708+0000 7fda3f718640 1 -- 172.21.15.92:0/1146327267 wait complete. 2024-09-11T22:13:06.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 -- 172.21.15.92:0/3577032116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 msgr2=0x7f3b6c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 --2- 172.21.15.92:0/3577032116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 0x7f3b6c101200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009900 tx=0x7f3b6002a480 comp rx=0 tx=0).stop 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 -- 172.21.15.92:0/3577032116 shutdown_connections 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 --2- 172.21.15.92:0/3577032116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 --2- 172.21.15.92:0/3577032116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b6c102570 0x7f3b6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 --2- 172.21.15.92:0/3577032116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 0x7f3b6c101200 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 -- 172.21.15.92:0/3577032116 >> 172.21.15.92:0/3577032116 conn(0x7f3b6c0fc7d0 msgr2=0x7f3b6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 -- 172.21.15.92:0/3577032116 shutdown_connections 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 -- 172.21.15.92:0/3577032116 wait complete. 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 Processor -- start 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.460+0000 7f3b73370640 1 -- start start 2024-09-11T22:13:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 0x7f3b6c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b6c102570 0x7f3b6c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c110e50 con 0x7f3b6c100e20 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c110cd0 con 0x7f3b6c102570 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c110fd0 con 0x7f3b6c102ef0 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 0x7f3b6c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 0x7f3b6c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51446/0 (socket says 172.21.15.92:51446) 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 -- 172.21.15.92:0/907260981 learned_addr learned my addr 172.21.15.92:0/907260981 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7090b640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b6c102570 0x7f3b6c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7190d640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 -- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c102ef0 msgr2=0x7f3b6c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 -- 172.21.15.92:0/907260981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b6c102570 msgr2=0x7f3b6c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b6c102570 0x7f3b6c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 -- 172.21.15.92:0/907260981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c19d480 con 0x7f3b6c100e20 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7090b640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b6c102570 0x7f3b6c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b7110c640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 0x7f3b6c1984d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b60006010 tx=0x7f3b600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6003f070 con 0x7f3b6c100e20 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c19d710 con 0x7f3b6c100e20 2024-09-11T22:13:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b60003aa0 con 0x7f3b6c100e20 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60035030 con 0x7f3b6c100e20 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.461+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c1a5050 con 0x7f3b6c100e20 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.462+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b6c062980 con 0x7f3b6c100e20 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.464+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3b60042070 con 0x7f3b6c100e20 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.464+0000 7f3b5e7fc640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b580a2c10 0x7f3b580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.464+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3b600f0480 con 0x7f3b6c100e20 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.464+0000 7f3b7090b640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b580a2c10 0x7f3b580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.464+0000 7f3b5e7fc640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] conn(0x7f3b580ad940 0x7f3b580afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.464+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 --> [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b60046e50 con 0x7f3b580ad940 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.464+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3b600f08d0 con 0x7f3b6c100e20 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.465+0000 7f3b7090b640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b580a2c10 0x7f3b580a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c1000e0 tx=0x7f3b68005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.465+0000 7f3b7190d640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] conn(0x7f3b580ad940 0x7f3b580afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.465+0000 7f3b7190d640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] conn(0x7f3b580ad940 0x7f3b580afd80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.466+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== osd.11 v2:172.21.15.153:6824/3726820847 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b60046e50 con 0x7f3b580ad940 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.480+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 --> [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b6c062d80 con 0x7f3b580ad940 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.480+0000 7f3b5e7fc640 1 -- 172.21.15.92:0/907260981 <== osd.11 v2:172.21.15.153:6824/3726820847 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b6c062d80 con 0x7f3b580ad940 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.480+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] conn(0x7f3b580ad940 msgr2=0x7f3b580afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] conn(0x7f3b580ad940 0x7f3b580afd80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b580a2c10 msgr2=0x7f3b580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b580a2c10 0x7f3b580a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c1000e0 tx=0x7f3b68005ec0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 msgr2=0x7f3b6c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 --2- 172.21.15.92:0/907260981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b6c100e20 0x7f3b6c1984d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b60006010 tx=0x7f3b600042f0 comp rx=0 tx=0).stop 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b7190d640 1 -- 172.21.15.92:0/907260981 reap_dead start 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 shutdown_connections 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 >> 172.21.15.92:0/907260981 conn(0x7f3b6c0fc7d0 msgr2=0x7f3b6c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 shutdown_connections 2024-09-11T22:13:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:05.481+0000 7f3b73370640 1 -- 172.21.15.92:0/907260981 wait complete. 2024-09-11T22:13:07.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:06 smithi092 ceph-mon[31173]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-11T22:13:07.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:06 smithi153 ceph-mon[35045]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-11T22:13:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:06 smithi110 ceph-mon[34830]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-11T22:13:07.451 INFO:teuthology.orchestra.run.smithi092.stdout:335007449113 2024-09-11T22:13:07.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T22:13:07.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1111455490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e18106f20 msgr2=0x7f3e1810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 --2- 172.21.15.92:0/1111455490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e18106f20 0x7f3e1810eaa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009ca0 tx=0x7f3e1002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1111455490 shutdown_connections 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 --2- 172.21.15.92:0/1111455490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e18106f20 0x7f3e1810eaa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 --2- 172.21.15.92:0/1111455490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e181027a0 0x7f3e181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 --2- 172.21.15.92:0/1111455490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e18101df0 0x7f3e181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1111455490 >> 172.21.15.92:0/1111455490 conn(0x7f3e180fd8f0 msgr2=0x7f3e180ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1111455490 shutdown_connections 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.590+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1111455490 wait complete. 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 Processor -- start 2024-09-11T22:13:07.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 -- start start 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e18101df0 0x7f3e181982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e181027a0 0x7f3e18198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e18106f20 0x7f3e1819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18111e30 con 0x7f3e181027a0 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18111cb0 con 0x7f3e18101df0 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1e3cf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18111fb0 con 0x7f3e18106f20 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e181027a0 0x7f3e18198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e181027a0 0x7f3e18198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51482/0 (socket says 172.21.15.92:51482) 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e177fe640 1 -- 172.21.15.92:0/1258890360 learned_addr learned my addr 172.21.15.92:0/1258890360 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e1c96c640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e18106f20 0x7f3e1819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.591+0000 7f3e17fff640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e18101df0 0x7f3e181982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e177fe640 1 -- 172.21.15.92:0/1258890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e18106f20 msgr2=0x7f3e1819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e177fe640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e18106f20 0x7f3e1819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e177fe640 1 -- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e18101df0 msgr2=0x7f3e181982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e177fe640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e18101df0 0x7f3e181982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e177fe640 1 -- 172.21.15.92:0/1258890360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1819d2a0 con 0x7f3e181027a0 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e17fff640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e18101df0 0x7f3e181982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e177fe640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e181027a0 0x7f3e18198830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00d770 tx=0x7f3e0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c017070 con 0x7f3e181027a0 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1819d590 con 0x7f3e181027a0 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e0c0054d0 con 0x7f3e181027a0 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004cf0 con 0x7f3e181027a0 2024-09-11T22:13:07.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.592+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e181a4e70 con 0x7f3e181027a0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.593+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e18062940 con 0x7f3e181027a0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.595+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3e0c004e90 con 0x7f3e181027a0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e157fa640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3e000a2c10 0x7f3e000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3e0c0cf220 con 0x7f3e181027a0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e17fff640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3e000a2c10 0x7f3e000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e157fa640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] conn(0x7f3e000ad940 0x7f3e000afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 --> [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e0c005320 con 0x7f3e000ad940 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c0cf670 con 0x7f3e181027a0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e17fff640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3e000a2c10 0x7f3e000a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800abe0 tx=0x7f3e0800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.596+0000 7f3e1c96c640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] conn(0x7f3e000ad940 0x7f3e000afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.597+0000 7f3e1c96c640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] conn(0x7f3e000ad940 0x7f3e000afd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.598+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== osd.4 v2:172.21.15.110:6808/712194146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e0c005320 con 0x7f3e000ad940 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.612+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 --> [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e18062dc0 con 0x7f3e000ad940 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e157fa640 1 -- 172.21.15.92:0/1258890360 <== osd.4 v2:172.21.15.110:6808/712194146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3e18062dc0 con 0x7f3e000ad940 2024-09-11T22:13:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] conn(0x7f3e000ad940 msgr2=0x7f3e000afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] conn(0x7f3e000ad940 0x7f3e000afd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3e000a2c10 msgr2=0x7f3e000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3e000a2c10 0x7f3e000a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800abe0 tx=0x7f3e0800a5c0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e181027a0 msgr2=0x7f3e18198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 --2- 172.21.15.92:0/1258890360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e181027a0 0x7f3e18198830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00d770 tx=0x7f3e0c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1c96c640 1 -- 172.21.15.92:0/1258890360 reap_dead start 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 shutdown_connections 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.613+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 >> 172.21.15.92:0/1258890360 conn(0x7f3e180fd8f0 msgr2=0x7f3e180fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.614+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 shutdown_connections 2024-09-11T22:13:07.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:06.614+0000 7f3e1e3cf640 1 -- 172.21.15.92:0/1258890360 wait complete. 2024-09-11T22:13:08.165 INFO:teuthology.orchestra.run.smithi092.stdout:150323855421 2024-09-11T22:13:08.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T22:13:08.839 INFO:teuthology.orchestra.run.smithi092.stdout:201863462962 2024-09-11T22:13:08.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T22:13:09.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:08 smithi092 ceph-mon[31173]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-11T22:13:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:08 smithi153 ceph-mon[35045]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-11T22:13:09.163 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:09.164 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:08 smithi110 ceph-mon[34830]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-11T22:13:09.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 -- 172.21.15.92:0/2086552256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af8103670 msgr2=0x7f6af810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:09.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 --2- 172.21.15.92:0/2086552256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af8103670 0x7f6af810bb60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009900 tx=0x7f6aec02a480 comp rx=0 tx=0).stop 2024-09-11T22:13:09.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 -- 172.21.15.92:0/2086552256 shutdown_connections 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 --2- 172.21.15.92:0/2086552256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af810c180 0x7f6af810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 --2- 172.21.15.92:0/2086552256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af8103670 0x7f6af810bb60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 --2- 172.21.15.92:0/2086552256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af8102d50 0x7f6af8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 -- 172.21.15.92:0/2086552256 >> 172.21.15.92:0/2086552256 conn(0x7f6af80fc7d0 msgr2=0x7f6af80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 -- 172.21.15.92:0/2086552256 shutdown_connections 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.597+0000 7f6afcbf8640 1 -- 172.21.15.92:0/2086552256 wait complete. 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 Processor -- start 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 -- start start 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af8102d50 0x7f6af8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af8103670 0x7f6af8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af810c180 0x7f6af819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8111e30 con 0x7f6af810c180 2024-09-11T22:13:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8111cb0 con 0x7f6af8103670 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6afcbf8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8111fb0 con 0x7f6af8102d50 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af810c180 0x7f6af819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af810c180 0x7f6af819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51522/0 (socket says 172.21.15.92:51522) 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 -- 172.21.15.92:0/869381887 learned_addr learned my addr 172.21.15.92:0/869381887 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af5d9b640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af8103670 0x7f6af8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af659c640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af8102d50 0x7f6af8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 -- 172.21.15.92:0/869381887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af8102d50 msgr2=0x7f6af8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af8102d50 0x7f6af8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 -- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af8103670 msgr2=0x7f6af8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6af8103670 0x7f6af8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.598+0000 7f6af6d9d640 1 -- 172.21.15.92:0/869381887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af819d3d0 con 0x7f6af810c180 2024-09-11T22:13:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.599+0000 7f6af6d9d640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af810c180 0x7f6af819ccf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6af000eaa0 tx=0x7f6af000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.599+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af000cdb0 con 0x7f6af810c180 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.599+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6af0005020 con 0x7f6af810c180 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.599+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af819d6c0 con 0x7f6af810c180 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.599+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af00116d0 con 0x7f6af810c180 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.599+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a4fa0 con 0x7f6af810c180 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.600+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6af8062900 con 0x7f6af810c180 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.601+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6af00051d0 con 0x7f6af810c180 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.601+0000 7f6ae77fe640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6af659c640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6af00cf930 con 0x7f6af810c180 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6ae77fe640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] conn(0x7f6ae00ad940 0x7f6ae00afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 --> [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6af0014ec0 con 0x7f6ae00ad940 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6af00cfd80 con 0x7f6af810c180 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6af659c640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005fd0 tx=0x7f6ae8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6af5d9b640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] conn(0x7f6ae00ad940 0x7f6ae00afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.602+0000 7f6af5d9b640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] conn(0x7f6ae00ad940 0x7f6ae00afd80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.603+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== osd.7 v2:172.21.15.110:6816/1905314527 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6af0014ec0 con 0x7f6ae00ad940 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.617+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 --> [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6af8062d80 con 0x7f6ae00ad940 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.617+0000 7f6ae77fe640 1 -- 172.21.15.92:0/869381887 <== osd.7 v2:172.21.15.110:6816/1905314527 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6af8062d80 con 0x7f6ae00ad940 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] conn(0x7f6ae00ad940 msgr2=0x7f6ae00afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] conn(0x7f6ae00ad940 0x7f6ae00afd80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae00a2c10 msgr2=0x7f6ae00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005fd0 tx=0x7f6ae8005850 comp rx=0 tx=0).stop 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af810c180 msgr2=0x7f6af819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 --2- 172.21.15.92:0/869381887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6af810c180 0x7f6af819ccf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6af000eaa0 tx=0x7f6af000ef60 comp rx=0 tx=0).stop 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6af6d9d640 1 -- 172.21.15.92:0/869381887 reap_dead start 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 shutdown_connections 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 >> 172.21.15.92:0/869381887 conn(0x7f6af80fc7d0 msgr2=0x7f6af80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 shutdown_connections 2024-09-11T22:13:09.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:09.618+0000 7f6afcbf8640 1 -- 172.21.15.92:0/869381887 wait complete. 2024-09-11T22:13:10.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 -- 172.21.15.92:0/2282725726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60106a80 msgr2=0x7fee60106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 --2- 172.21.15.92:0/2282725726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60106a80 0x7fee60106e60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee54009930 tx=0x7fee5402a420 comp rx=0 tx=0).stop 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 -- 172.21.15.92:0/2282725726 shutdown_connections 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 --2- 172.21.15.92:0/2282725726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee60101420 0x7fee6010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 --2- 172.21.15.92:0/2282725726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee60100a80 0x7fee60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 --2- 172.21.15.92:0/2282725726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60106a80 0x7fee60106e60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 -- 172.21.15.92:0/2282725726 >> 172.21.15.92:0/2282725726 conn(0x7fee600fc7a0 msgr2=0x7fee600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 -- 172.21.15.92:0/2282725726 shutdown_connections 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 -- 172.21.15.92:0/2282725726 wait complete. 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.374+0000 7fee687ae640 1 Processor -- start 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 -- start start 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee60100a80 0x7fee600fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60101420 0x7fee600fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee60106a80 0x7fee600fd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee60110dd0 con 0x7fee60101420 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee60110c50 con 0x7fee60100a80 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee60110f50 con 0x7fee60106a80 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60101420 0x7fee600fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60101420 0x7fee600fd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51536/0 (socket says 172.21.15.92:51536) 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 -- 172.21.15.92:0/832974307 learned_addr learned my addr 172.21.15.92:0/832974307 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 -- 172.21.15.92:0/832974307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee60106a80 msgr2=0x7fee600fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee66d4b640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee60106a80 0x7fee600fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee60106a80 0x7fee600fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee6654a640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee60100a80 0x7fee600fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 -- 172.21.15.92:0/832974307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee60100a80 msgr2=0x7fee600fcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee60100a80 0x7fee600fcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 -- 172.21.15.92:0/832974307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee601a6a00 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee66d4b640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee60106a80 0x7fee600fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee65d49640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60101420 0x7fee600fd420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00b4a0 tx=0x7fee5c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c014070 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee601a6cf0 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.375+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee5c010ac0 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.376+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c017c30 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.376+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a7180 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.376+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee600628c0 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.378+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee5c017070 con 0x7fee60101420 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.378+0000 7fee537fe640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fee4c0a2c10 0x7fee4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee6654a640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fee4c0a2c10 0x7fee4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fee5c0ceb60 con 0x7fee60101420 2024-09-11T22:13:10.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee537fe640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] conn(0x7fee4c0ad940 0x7fee4c0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 --> [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee5c01be50 con 0x7fee4c0ad940 2024-09-11T22:13:10.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fee5c0cefb0 con 0x7fee60101420 2024-09-11T22:13:10.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee66d4b640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] conn(0x7fee4c0ad940 0x7fee4c0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee6654a640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fee4c0a2c10 0x7fee4c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee54006010 tx=0x7fee54000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.379+0000 7fee66d4b640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] conn(0x7fee4c0ad940 0x7fee4c0afd80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.380+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== osd.9 v2:172.21.15.92:6826/3782209439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee5c01be50 con 0x7fee4c0ad940 2024-09-11T22:13:10.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 --> [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee60062d40 con 0x7fee4c0ad940 2024-09-11T22:13:10.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee537fe640 1 -- 172.21.15.92:0/832974307 <== osd.9 v2:172.21.15.92:6826/3782209439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fee60062d40 con 0x7fee4c0ad940 2024-09-11T22:13:10.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] conn(0x7fee4c0ad940 msgr2=0x7fee4c0afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] conn(0x7fee4c0ad940 0x7fee4c0afd80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fee4c0a2c10 msgr2=0x7fee4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fee4c0a2c10 0x7fee4c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee54006010 tx=0x7fee54000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60101420 msgr2=0x7fee600fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 --2- 172.21.15.92:0/832974307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee60101420 0x7fee600fd420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00b4a0 tx=0x7fee5c00b960 comp rx=0 tx=0).stop 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee66d4b640 1 -- 172.21.15.92:0/832974307 reap_dead start 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 shutdown_connections 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 >> 172.21.15.92:0/832974307 conn(0x7fee600fc7a0 msgr2=0x7fee6006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 shutdown_connections 2024-09-11T22:13:10.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.394+0000 7fee687ae640 1 -- 172.21.15.92:0/832974307 wait complete. 2024-09-11T22:13:10.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.883+0000 7eff28426640 1 -- 172.21.15.92:0/424496754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff20103760 msgr2=0x7eff20107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.883+0000 7eff28426640 1 --2- 172.21.15.92:0/424496754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff20103760 0x7eff20107ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7eff10009930 tx=0x7eff1002a420 comp rx=0 tx=0).stop 2024-09-11T22:13:10.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.883+0000 7eff28426640 1 -- 172.21.15.92:0/424496754 shutdown_connections 2024-09-11T22:13:10.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 --2- 172.21.15.92:0/424496754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff201082e0 0x7eff2010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 --2- 172.21.15.92:0/424496754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff20103760 0x7eff20107ba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 --2- 172.21.15.92:0/424496754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff20102db0 0x7eff20103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 -- 172.21.15.92:0/424496754 >> 172.21.15.92:0/424496754 conn(0x7eff200fc7d0 msgr2=0x7eff200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 -- 172.21.15.92:0/424496754 shutdown_connections 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 -- 172.21.15.92:0/424496754 wait complete. 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 Processor -- start 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 -- start start 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff20102db0 0x7eff20198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff20103760 0x7eff20198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff201082e0 0x7eff2019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7eff20111f00 con 0x7eff201082e0 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7eff20111d80 con 0x7eff20103760 2024-09-11T22:13:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff28426640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff20112080 con 0x7eff20102db0 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff269c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff201082e0 0x7eff2019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff269c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff201082e0 0x7eff2019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51556/0 (socket says 172.21.15.92:51556) 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.884+0000 7eff269c3640 1 -- 172.21.15.92:0/2144136270 learned_addr learned my addr 172.21.15.92:0/2144136270 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff259c1640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff20103760 0x7eff20198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff261c2640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff20102db0 0x7eff20198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff269c3640 1 -- 172.21.15.92:0/2144136270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff20102db0 msgr2=0x7eff20198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff269c3640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff20102db0 0x7eff20198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff269c3640 1 -- 172.21.15.92:0/2144136270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff20103760 msgr2=0x7eff20198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff269c3640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff20103760 0x7eff20198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff269c3640 1 -- 172.21.15.92:0/2144136270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2019d4c0 con 0x7eff201082e0 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff259c1640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff20103760 0x7eff20198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff269c3640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff201082e0 0x7eff2019cde0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7eff1800eaa0 tx=0x7eff1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1800cdb0 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2019d7b0 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff18005020 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180116d0 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.885+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff201a5090 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.886+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff20062980 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.887+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff180051d0 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff1f7fe640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7eff080a2c10 0x7eff080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff261c2640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7eff080a2c10 0x7eff080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7eff180cf670 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff1f7fe640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] conn(0x7eff080ad940 0x7eff080afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 --> [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff18018ec0 con 0x7eff080ad940 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7eff180d6050 con 0x7eff201082e0 2024-09-11T22:13:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff259c1640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] conn(0x7eff080ad940 0x7eff080afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff261c2640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7eff080a2c10 0x7eff080a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7eff140058c0 tx=0x7eff1400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.888+0000 7eff259c1640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] conn(0x7eff080ad940 0x7eff080afd80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.889+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== osd.3 v2:172.21.15.92:6810/931675439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff18018ec0 con 0x7eff080ad940 2024-09-11T22:13:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.902+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 --> [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff20062e00 con 0x7eff080ad940 2024-09-11T22:13:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff1f7fe640 1 -- 172.21.15.92:0/2144136270 <== osd.3 v2:172.21.15.92:6810/931675439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7eff20062e00 con 0x7eff080ad940 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] conn(0x7eff080ad940 msgr2=0x7eff080afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] conn(0x7eff080ad940 0x7eff080afd80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7eff080a2c10 msgr2=0x7eff080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7eff080a2c10 0x7eff080a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7eff140058c0 tx=0x7eff1400a4c0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff201082e0 msgr2=0x7eff2019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 --2- 172.21.15.92:0/2144136270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff201082e0 0x7eff2019cde0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7eff1800eaa0 tx=0x7eff1800ef60 comp rx=0 tx=0).stop 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff269c3640 1 -- 172.21.15.92:0/2144136270 reap_dead start 2024-09-11T22:13:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 shutdown_connections 2024-09-11T22:13:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 >> 172.21.15.92:0/2144136270 conn(0x7eff200fc7d0 msgr2=0x7eff20105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 shutdown_connections 2024-09-11T22:13:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:10.903+0000 7eff28426640 1 -- 172.21.15.92:0/2144136270 wait complete. 2024-09-11T22:13:11.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:10 smithi092 ceph-mon[31173]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T22:13:11.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:10 smithi153 ceph-mon[35045]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T22:13:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:10 smithi110 ceph-mon[34830]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T22:13:11.259 INFO:teuthology.orchestra.run.smithi092.stdout:253403070505 2024-09-11T22:13:11.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T22:13:11.925 INFO:teuthology.orchestra.run.smithi092.stdout:223338299439 2024-09-11T22:13:11.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T22:13:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.201+0000 7f07431fe640 1 -- 172.21.15.92:0/2003599547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 msgr2=0x7f073c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.201+0000 7f07431fe640 1 --2- 172.21.15.92:0/2003599547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 0x7f073c100ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0728009930 tx=0x7f072802a420 comp rx=0 tx=0).stop 2024-09-11T22:13:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.202+0000 7f07431fe640 1 -- 172.21.15.92:0/2003599547 shutdown_connections 2024-09-11T22:13:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.202+0000 7f07431fe640 1 --2- 172.21.15.92:0/2003599547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f073c101420 0x7f073c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.202+0000 7f07431fe640 1 --2- 172.21.15.92:0/2003599547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 0x7f073c100ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.202+0000 7f07431fe640 1 --2- 172.21.15.92:0/2003599547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f073c106a80 0x7f073c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.202+0000 7f07431fe640 1 -- 172.21.15.92:0/2003599547 >> 172.21.15.92:0/2003599547 conn(0x7f073c0fc7a0 msgr2=0x7f073c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.202+0000 7f07431fe640 1 -- 172.21.15.92:0/2003599547 shutdown_connections 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.202+0000 7f07431fe640 1 -- 172.21.15.92:0/2003599547 wait complete. 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 Processor -- start 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 -- start start 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 0x7f073c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f073c101420 0x7f073c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f073c106a80 0x7f073c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f073c110d80 con 0x7f073c100a80 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f073c110c00 con 0x7f073c106a80 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f07431fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f073c110f00 con 0x7f073c101420 2024-09-11T22:13:12.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 0x7f073c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 0x7f073c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51566/0 (socket says 172.21.15.92:51566) 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 -- 172.21.15.92:0/2875180559 learned_addr learned my addr 172.21.15.92:0/2875180559 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f073bfff640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f073c101420 0x7f073c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f074179b640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f073c106a80 0x7f073c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 -- 172.21.15.92:0/2875180559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f073c101420 msgr2=0x7f073c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f073c101420 0x7f073c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 -- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f073c106a80 msgr2=0x7f073c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f073c106a80 0x7f073c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.203+0000 7f0740f9a640 1 -- 172.21.15.92:0/2875180559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c19c1f0 con 0x7f073c100a80 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.204+0000 7f0740f9a640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 0x7f073c197240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0730007b50 tx=0x7f073000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.204+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730007c40 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.204+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0730007de0 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.204+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073000ad40 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.204+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c19c4e0 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.204+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c06a790 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.205+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f073c06ad40 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.206+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f073000eb30 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f0739ffb640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f072c0a2c90 0x7f072c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f073bfff640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f072c0a2c90 0x7f072c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f07300d0830 con 0x7f073c100a80 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f0739ffb640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] conn(0x7f072c0ad9c0 0x7f072c0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 --> [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f072c0b04d0 con 0x7f072c0ad9c0 2024-09-11T22:13:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f07300d52c0 con 0x7f073c100a80 2024-09-11T22:13:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f073bfff640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f072c0a2c90 0x7f072c0a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0728031440 tx=0x7f0728005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.207+0000 7f074179b640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] conn(0x7f072c0ad9c0 0x7f072c0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.208+0000 7f074179b640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] conn(0x7f072c0ad9c0 0x7f072c0afe00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.208+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== osd.1 v2:172.21.15.110:6800/1945048542 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f072c0b04d0 con 0x7f072c0ad9c0 2024-09-11T22:13:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.222+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 --> [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f073c06b210 con 0x7f072c0ad9c0 2024-09-11T22:13:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f0739ffb640 1 -- 172.21.15.92:0/2875180559 <== osd.1 v2:172.21.15.110:6800/1945048542 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f073c06b210 con 0x7f072c0ad9c0 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] conn(0x7f072c0ad9c0 msgr2=0x7f072c0afe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] conn(0x7f072c0ad9c0 0x7f072c0afe00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f072c0a2c90 msgr2=0x7f072c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f072c0a2c90 0x7f072c0a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0728031440 tx=0x7f0728005a20 comp rx=0 tx=0).stop 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 msgr2=0x7f073c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 --2- 172.21.15.92:0/2875180559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f073c100a80 0x7f073c197240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0730007b50 tx=0x7f073000e420 comp rx=0 tx=0).stop 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f074179b640 1 -- 172.21.15.92:0/2875180559 reap_dead start 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 shutdown_connections 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 >> 172.21.15.92:0/2875180559 conn(0x7f073c0fc7a0 msgr2=0x7f073c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 shutdown_connections 2024-09-11T22:13:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:12.223+0000 7f07431fe640 1 -- 172.21.15.92:0/2875180559 wait complete. 2024-09-11T22:13:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:12 smithi092 ceph-mon[31173]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T22:13:13.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:12 smithi153 ceph-mon[35045]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T22:13:13.132 INFO:teuthology.orchestra.run.smithi092.stdout:55834574925 2024-09-11T22:13:13.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T22:13:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:12 smithi110 ceph-mon[34830]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T22:13:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 -- 172.21.15.92:0/1049707185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 msgr2=0x7f46ac1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 --2- 172.21.15.92:0/1049707185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 0x7f46ac1067d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4698009900 tx=0x7f469802a410 comp rx=0 tx=0).stop 2024-09-11T22:13:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 -- 172.21.15.92:0/1049707185 shutdown_connections 2024-09-11T22:13:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 --2- 172.21.15.92:0/1049707185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ac106f00 0x7f46ac10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 --2- 172.21.15.92:0/1049707185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 0x7f46ac1067d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 --2- 172.21.15.92:0/1049707185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac101dd0 0x7f46ac1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 -- 172.21.15.92:0/1049707185 >> 172.21.15.92:0/1049707185 conn(0x7f46ac0fd8b0 msgr2=0x7f46ac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 -- 172.21.15.92:0/1049707185 shutdown_connections 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.272+0000 7f46b3249640 1 -- 172.21.15.92:0/1049707185 wait complete. 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 Processor -- start 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 -- start start 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ac101dd0 0x7f46ac1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 0x7f46ac198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac106f00 0x7f46ac19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac111e30 con 0x7f46ac102780 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac111cb0 con 0x7f46ac106f00 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b3249640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac111fb0 con 0x7f46ac101dd0 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46abfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 0x7f46ac198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46abfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 0x7f46ac198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51590/0 (socket says 172.21.15.92:51590) 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46abfff640 1 -- 172.21.15.92:0/2542009092 learned_addr learned my addr 172.21.15.92:0/2542009092 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b0fe5640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ac101dd0 0x7f46ac1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.273+0000 7f46b17e6640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac106f00 0x7f46ac19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46abfff640 1 -- 172.21.15.92:0/2542009092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ac101dd0 msgr2=0x7f46ac1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46abfff640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ac101dd0 0x7f46ac1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46abfff640 1 -- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac106f00 msgr2=0x7f46ac19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46abfff640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac106f00 0x7f46ac19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46abfff640 1 -- 172.21.15.92:0/2542009092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac19d4a0 con 0x7f46ac102780 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46abfff640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 0x7f46ac198a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4698006010 tx=0x7f4698004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469803f030 con 0x7f46ac102780 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46980035f0 con 0x7f46ac102780 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac19d730 con 0x7f46ac102780 2024-09-11T22:13:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698035070 con 0x7f46ac102780 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.274+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac1a5070 con 0x7f46ac102780 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.275+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46ac062900 con 0x7f46ac102780 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.277+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4698003b10 con 0x7f46ac102780 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.277+0000 7f46a9ffb640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f469c0a2c10 0x7f469c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.277+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f46980f05a0 con 0x7f46ac102780 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.277+0000 7f46b0fe5640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f469c0a2c10 0x7f469c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.278+0000 7f46a9ffb640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] conn(0x7f469c0ad940 0x7f469c0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.278+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 --> [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4698001d30 con 0x7f469c0ad940 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.278+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f46980f09f0 con 0x7f46ac102780 2024-09-11T22:13:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.278+0000 7f46b0fe5640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f469c0a2c10 0x7f469c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46a00058c0 tx=0x7f46a000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.278+0000 7f46b17e6640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] conn(0x7f469c0ad940 0x7f469c0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.278+0000 7f46b17e6640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] conn(0x7f469c0ad940 0x7f469c0afd80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.279+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== osd.10 v2:172.21.15.110:6824/3399897751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4698001d30 con 0x7f469c0ad940 2024-09-11T22:13:13.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.293+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 --> [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46ac062d00 con 0x7f469c0ad940 2024-09-11T22:13:13.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.293+0000 7f46a9ffb640 1 -- 172.21.15.92:0/2542009092 <== osd.10 v2:172.21.15.110:6824/3399897751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46ac062d00 con 0x7f469c0ad940 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] conn(0x7f469c0ad940 msgr2=0x7f469c0afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] conn(0x7f469c0ad940 0x7f469c0afd80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f469c0a2c10 msgr2=0x7f469c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f469c0a2c10 0x7f469c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46a00058c0 tx=0x7f46a000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 msgr2=0x7f46ac198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 --2- 172.21.15.92:0/2542009092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46ac102780 0x7f46ac198a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4698006010 tx=0x7f4698004270 comp rx=0 tx=0).stop 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b17e6640 1 -- 172.21.15.92:0/2542009092 reap_dead start 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 shutdown_connections 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 >> 172.21.15.92:0/2542009092 conn(0x7f46ac0fd8b0 msgr2=0x7f46ac0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 shutdown_connections 2024-09-11T22:13:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:13.294+0000 7f46b3249640 1 -- 172.21.15.92:0/2542009092 wait complete. 2024-09-11T22:13:14.071 INFO:teuthology.orchestra.run.smithi092.stdout:115964117059 2024-09-11T22:13:14.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T22:13:14.371 INFO:teuthology.orchestra.run.smithi092.stdout:283467841572 2024-09-11T22:13:14.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T22:13:14.830 INFO:teuthology.orchestra.run.smithi092.stdout:304942678047 2024-09-11T22:13:14.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T22:13:14.925 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:14.926 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:14 smithi092 ceph-mon[31173]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T22:13:15.036 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:15.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:14 smithi153 ceph-mon[35045]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T22:13:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:14 smithi110 ceph-mon[34830]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T22:13:15.669 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:15.675 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:15.675 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:15.685 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-11T22:13:15.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:15 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-11T22:13:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 -- 172.21.15.92:0/3747308091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 msgr2=0x7f28d010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 --2- 172.21.15.92:0/3747308091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 0x7f28d010ea20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009ca0 tx=0x7f28c802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:13:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 -- 172.21.15.92:0/3747308091 shutdown_connections 2024-09-11T22:13:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 --2- 172.21.15.92:0/3747308091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 0x7f28d010ea20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 --2- 172.21.15.92:0/3747308091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28d0100100 0x7f28d0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 --2- 172.21.15.92:0/3747308091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0106b80 0x7f28d0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 -- 172.21.15.92:0/3747308091 >> 172.21.15.92:0/3747308091 conn(0x7f28d0077230 msgr2=0x7f28d00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 -- 172.21.15.92:0/3747308091 shutdown_connections 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 -- 172.21.15.92:0/3747308091 wait complete. 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.963+0000 7f28d5057640 1 Processor -- start 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28d5057640 1 -- start start 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28d5057640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0100100 0x7f28d019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28d5057640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 0x7f28d019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28d5057640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28d0106b80 0x7f28d019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28d5057640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0111f40 con 0x7f28d0100aa0 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28d5057640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0111dc0 con 0x7f28d0100100 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28d5057640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01120c0 con 0x7f28d0106b80 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28ce59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 0x7f28d019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28ce59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 0x7f28d019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39664/0 (socket says 172.21.15.92:39664) 2024-09-11T22:13:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28ce59c640 1 -- 172.21.15.92:0/852220358 learned_addr learned my addr 172.21.15.92:0/852220358 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28ced9d640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0100100 0x7f28d019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.964+0000 7f28cf59e640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28d0106b80 0x7f28d019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28ce59c640 1 -- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28d0106b80 msgr2=0x7f28d019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28ce59c640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28d0106b80 0x7f28d019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28ce59c640 1 -- 172.21.15.92:0/852220358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0100100 msgr2=0x7f28d019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28ce59c640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0100100 0x7f28d019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28ce59c640 1 -- 172.21.15.92:0/852220358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d019f5f0 con 0x7f28d0100aa0 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28cf59e640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28d0106b80 0x7f28d019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28ce59c640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 0x7f28d019ab80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c970 tx=0x7f28c000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0007bb0 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d019f8e0 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28c0007d50 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0004e50 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.965+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01a71c0 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.966+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28d010e1a0 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.967+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f28c0005000 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.968+0000 7f28b7fff640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f28b80a2db0 0x7f28b80a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.968+0000 7f28ced9d640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f28b80a2db0 0x7f28b80a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.968+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f28c00cf6b0 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.968+0000 7f28b7fff640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] conn(0x7f28b80adf80 0x7f28b80b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.968+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 --> [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28c0022e50 con 0x7f28b80adf80 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.968+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f28c00cfb00 con 0x7f28d0100aa0 2024-09-11T22:13:15.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.968+0000 7f28ced9d640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f28b80a2db0 0x7f28b80a5270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009990 tx=0x7f28c4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.969+0000 7f28cf59e640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] conn(0x7f28b80adf80 0x7f28b80b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.969+0000 7f28cf59e640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] conn(0x7f28b80adf80 0x7f28b80b0420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.970+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== osd.2 v2:172.21.15.153:6800/2033986544 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28c0022e50 con 0x7f28b80adf80 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 --> [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28d0103420 con 0x7f28b80adf80 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28b7fff640 1 -- 172.21.15.92:0/852220358 <== osd.2 v2:172.21.15.153:6800/2033986544 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f28d0103420 con 0x7f28b80adf80 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] conn(0x7f28b80adf80 msgr2=0x7f28b80b0420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] conn(0x7f28b80adf80 0x7f28b80b0420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f28b80a2db0 msgr2=0x7f28b80a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f28b80a2db0 0x7f28b80a5270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009990 tx=0x7f28c4009110 comp rx=0 tx=0).stop 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 msgr2=0x7f28d019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 --2- 172.21.15.92:0/852220358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0100aa0 0x7f28d019ab80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c970 tx=0x7f28c000cd70 comp rx=0 tx=0).stop 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28cf59e640 1 -- 172.21.15.92:0/852220358 reap_dead start 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 shutdown_connections 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.984+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 >> 172.21.15.92:0/852220358 conn(0x7f28d0077230 msgr2=0x7f28d0104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:15.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.985+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 shutdown_connections 2024-09-11T22:13:16.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:15.985+0000 7f28d5057640 1 -- 172.21.15.92:0/852220358 wait complete. 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-11T22:13:16.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:15 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-11T22:13:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-11T22:13:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-11T22:13:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-11T22:13:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T22:13:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-11T22:13:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T22:13:16.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-09-11T22:13:16.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-11T22:13:16.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-11T22:13:16.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:15 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-11T22:13:16.624 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:16.660 INFO:teuthology.orchestra.run.smithi092.stdout:77309411401 2024-09-11T22:13:16.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T22:13:16.910 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T22:13:16.910 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-11T22:13:16.910 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-11T22:13:16.910 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-11T22:13:16.911 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:16 smithi092 ceph-mon[31173]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T22:13:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-09-11T22:13:17.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-11T22:13:17.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-11T22:13:17.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-11T22:13:17.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:16 smithi153 ceph-mon[35045]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T22:13:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-09-11T22:13:17.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-11T22:13:17.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-11T22:13:17.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-11T22:13:17.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:16 smithi110 ceph-mon[34830]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:13:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 -- 172.21.15.92:0/228663062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850106a80 msgr2=0x7fb850106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 --2- 172.21.15.92:0/228663062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850106a80 0x7fb850106e60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009930 tx=0x7fb84c02a420 comp rx=0 tx=0).stop 2024-09-11T22:13:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 -- 172.21.15.92:0/228663062 shutdown_connections 2024-09-11T22:13:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 --2- 172.21.15.92:0/228663062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb850101420 0x7fb85010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 --2- 172.21.15.92:0/228663062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb850100a80 0x7fb850100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 --2- 172.21.15.92:0/228663062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850106a80 0x7fb850106e60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 -- 172.21.15.92:0/228663062 >> 172.21.15.92:0/228663062 conn(0x7fb8500fc7a0 msgr2=0x7fb8500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 -- 172.21.15.92:0/228663062 shutdown_connections 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.662+0000 7fb858614640 1 -- 172.21.15.92:0/228663062 wait complete. 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 Processor -- start 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 -- start start 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb850100a80 0x7fb850197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850101420 0x7fb850197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb850106a80 0x7fb85019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb850110d50 con 0x7fb850101420 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb850110bd0 con 0x7fb850106a80 2024-09-11T22:13:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb858614640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb850110ed0 con 0x7fb850100a80 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850101420 0x7fb850197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850101420 0x7fb850197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39676/0 (socket says 172.21.15.92:39676) 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 -- 172.21.15.92:0/794132792 learned_addr learned my addr 172.21.15.92:0/794132792 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb856bb1640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb850106a80 0x7fb85019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb8563b0640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb850100a80 0x7fb850197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 -- 172.21.15.92:0/794132792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb850100a80 msgr2=0x7fb850197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb850100a80 0x7fb850197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 -- 172.21.15.92:0/794132792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb850106a80 msgr2=0x7fb85019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb850106a80 0x7fb85019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 -- 172.21.15.92:0/794132792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85019c1f0 con 0x7fb850101420 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb856bb1640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb850106a80 0x7fb85019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.663+0000 7fb855baf640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850101420 0x7fb850197780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb84000c920 tx=0x7fb84000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.664+0000 7fb8477fe640 1 -- 172.21.15.92:0/794132792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840007c70 con 0x7fb850101420 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.664+0000 7fb8477fe640 1 -- 172.21.15.92:0/794132792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb840007e10 con 0x7fb850101420 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.664+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85019c4e0 con 0x7fb850101420 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.664+0000 7fb8477fe640 1 -- 172.21.15.92:0/794132792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840018820 con 0x7fb850101420 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.664+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8501a3dc0 con 0x7fb850101420 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.665+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb850100ee0 con 0x7fb850101420 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.666+0000 7fb8477fe640 1 -- 172.21.15.92:0/794132792 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb8400189c0 con 0x7fb850101420 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.666+0000 7fb8477fe640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb83c0a2c90 0x7fb83c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:17.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.666+0000 7fb8563b0640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb83c0a2c90 0x7fb83c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:17.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.666+0000 7fb8477fe640 1 -- 172.21.15.92:0/794132792 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb8400cf6d0 con 0x7fb850101420 2024-09-11T22:13:17.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.667+0000 7fb8563b0640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb83c0a2c90 0x7fb83c0a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:17.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.671+0000 7fb8477fe640 1 -- 172.21.15.92:0/794132792 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb840014030 con 0x7fb850101420 2024-09-11T22:13:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.844+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb85002d7c0 con 0x7fb850101420 2024-09-11T22:13:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.845+0000 7fb8477fe640 1 -- 172.21.15.92:0/794132792 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb84009c170 con 0x7fb850101420 2024-09-11T22:13:17.845 INFO:teuthology.orchestra.run.smithi092.stdout:283467841573 2024-09-11T22:13:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb83c0a2c90 msgr2=0x7fb83c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb83c0a2c90 0x7fb83c0a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c031400 comp rx=0 tx=0).stop 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850101420 msgr2=0x7fb850197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850101420 0x7fb850197780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb84000c920 tx=0x7fb84000cde0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 shutdown_connections 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb83c0a2c90 0x7fb83c0a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb850106a80 0x7fb85019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb850101420 0x7fb850197780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 --2- 172.21.15.92:0/794132792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb850100a80 0x7fb850197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 >> 172.21.15.92:0/794132792 conn(0x7fb8500fc7a0 msgr2=0x7fb8500fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 shutdown_connections 2024-09-11T22:13:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.848+0000 7fb858614640 1 -- 172.21.15.92:0/794132792 wait complete. 2024-09-11T22:13:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:17 smithi153 ceph-mon[35045]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:13:18.077 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:17 smithi092 ceph-mon[31173]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:13:18.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 -- 172.21.15.92:0/539469960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 msgr2=0x7f6e2010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 --2- 172.21.15.92:0/539469960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 0x7f6e2010eaa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009ca0 tx=0x7f6e1002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 -- 172.21.15.92:0/539469960 shutdown_connections 2024-09-11T22:13:18.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 --2- 172.21.15.92:0/539469960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 0x7f6e2010eaa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 --2- 172.21.15.92:0/539469960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e201027a0 0x7f6e201067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 --2- 172.21.15.92:0/539469960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e20101df0 0x7f6e201021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 -- 172.21.15.92:0/539469960 >> 172.21.15.92:0/539469960 conn(0x7f6e200fd8b0 msgr2=0x7f6e200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 -- 172.21.15.92:0/539469960 shutdown_connections 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.915+0000 7f6e25127640 1 -- 172.21.15.92:0/539469960 wait complete. 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.916+0000 7f6e25127640 1 Processor -- start 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.916+0000 7f6e25127640 1 -- start start 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.916+0000 7f6e25127640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e20101df0 0x7f6e20198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e25127640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e201027a0 0x7f6e201988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e25127640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 0x7f6e2019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e25127640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20111e50 con 0x7f6e20106f20 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e25127640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20111cd0 con 0x7f6e20101df0 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e25127640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20111fd0 con 0x7f6e201027a0 2024-09-11T22:13:18.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 0x7f6e2019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 0x7f6e2019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39698/0 (socket says 172.21.15.92:39698) 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 -- 172.21.15.92:0/2972736844 learned_addr learned my addr 172.21.15.92:0/2972736844 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1e59c640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e201027a0 0x7f6e201988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1ed9d640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e20101df0 0x7f6e20198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 -- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e201027a0 msgr2=0x7f6e201988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e201027a0 0x7f6e201988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 -- 172.21.15.92:0/2972736844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e20101df0 msgr2=0x7f6e20198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e20101df0 0x7f6e20198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 -- 172.21.15.92:0/2972736844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2019d320 con 0x7f6e20106f20 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1ed9d640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e20101df0 0x7f6e20198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e1f59e640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 0x7f6e2019cc40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009dc0 tx=0x7f6e10032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10003cf0 con 0x7f6e20106f20 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e100044f0 con 0x7f6e20106f20 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2019d5b0 con 0x7f6e20106f20 2024-09-11T22:13:18.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10030b80 con 0x7f6e20106f20 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.917+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a4ef0 con 0x7f6e20106f20 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.918+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e20062980 con 0x7f6e20106f20 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.919+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6e100046a0 con 0x7f6e20106f20 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.920+0000 7f6e07fff640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6e080a2c10 0x7f6e080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.920+0000 7f6e1ed9d640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6e080a2c10 0x7f6e080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.920+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6e100f0c20 con 0x7f6e20106f20 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.920+0000 7f6e07fff640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] conn(0x7f6e080adde0 0x7f6e080b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.920+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 --> [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e1000b150 con 0x7f6e080adde0 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.920+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6e100f1070 con 0x7f6e20106f20 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.920+0000 7f6e1ed9d640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6e080a2c10 0x7f6e080a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e180058c0 tx=0x7f6e18005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.921+0000 7f6e1e59c640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] conn(0x7f6e080adde0 0x7f6e080b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.921+0000 7f6e1e59c640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] conn(0x7f6e080adde0 0x7f6e080b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.922+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== osd.5 v2:172.21.15.153:6808/1354584850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e1000b150 con 0x7f6e080adde0 2024-09-11T22:13:18.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.935+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 --> [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e20062e00 con 0x7f6e080adde0 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e07fff640 1 -- 172.21.15.92:0/2972736844 <== osd.5 v2:172.21.15.153:6808/1354584850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6e20062e00 con 0x7f6e080adde0 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] conn(0x7f6e080adde0 msgr2=0x7f6e080b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] conn(0x7f6e080adde0 0x7f6e080b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6e080a2c10 msgr2=0x7f6e080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6e080a2c10 0x7f6e080a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e180058c0 tx=0x7f6e18005850 comp rx=0 tx=0).stop 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 msgr2=0x7f6e2019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 --2- 172.21.15.92:0/2972736844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e20106f20 0x7f6e2019cc40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009dc0 tx=0x7f6e10032ff0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e1f59e640 1 -- 172.21.15.92:0/2972736844 reap_dead start 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 shutdown_connections 2024-09-11T22:13:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 >> 172.21.15.92:0/2972736844 conn(0x7f6e200fd8b0 msgr2=0x7f6e200febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:18.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 shutdown_connections 2024-09-11T22:13:18.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:17.936+0000 7f6e25127640 1 -- 172.21.15.92:0/2972736844 wait complete. 2024-09-11T22:13:18.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:17 smithi110 ceph-mon[34830]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:13:18.395 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:18.396 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:18.396 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:18.823 INFO:teuthology.orchestra.run.smithi092.stdout:171798691897 2024-09-11T22:13:18.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T22:13:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:18 smithi153 ceph-mon[35045]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-09-11T22:13:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:18 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/794132792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:13:19.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 -- 172.21.15.92:0/483458498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d681008c0 msgr2=0x7f7d68100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/483458498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d681008c0 0x7f7d68100ca0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009900 tx=0x7f7d6402a410 comp rx=0 tx=0).stop 2024-09-11T22:13:19.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 -- 172.21.15.92:0/483458498 shutdown_connections 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/483458498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d68105ec0 0x7f7d68106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/483458498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d681011e0 0x7f7d68105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/483458498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d681008c0 0x7f7d68100ca0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 -- 172.21.15.92:0/483458498 >> 172.21.15.92:0/483458498 conn(0x7f7d680fc7d0 msgr2=0x7f7d680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.018+0000 7f7d6fab1640 1 -- 172.21.15.92:0/483458498 shutdown_connections 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.019+0000 7f7d6fab1640 1 -- 172.21.15.92:0/483458498 wait complete. 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.019+0000 7f7d6fab1640 1 Processor -- start 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.019+0000 7f7d6fab1640 1 -- start start 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.019+0000 7f7d6fab1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d681008c0 0x7f7d681a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.019+0000 7f7d6fab1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d681011e0 0x7f7d681a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6fab1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d68105ec0 0x7f7d6819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6fab1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68118540 con 0x7f7d68105ec0 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6fab1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d681183c0 con 0x7f7d681011e0 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6fab1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d681186c0 con 0x7f7d681008c0 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d68105ec0 0x7f7d6819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d68105ec0 0x7f7d6819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39726/0 (socket says 172.21.15.92:39726) 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 -- 172.21.15.92:0/700826218 learned_addr learned my addr 172.21.15.92:0/700826218 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6d04c640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d681011e0 0x7f7d681a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 -- 172.21.15.92:0/700826218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d681008c0 msgr2=0x7f7d681a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d681008c0 0x7f7d681a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 -- 172.21.15.92:0/700826218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d681011e0 msgr2=0x7f7d681a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d681011e0 0x7f7d681a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 -- 172.21.15.92:0/700826218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6819eff0 con 0x7f7d68105ec0 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6d04c640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d681011e0 0x7f7d681a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:19.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6e04e640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d68105ec0 0x7f7d6819e850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000b970 tx=0x7f7d6000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d5effd640 1 -- 172.21.15.92:0/700826218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60019070 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d5effd640 1 -- 172.21.15.92:0/700826218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d60014440 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d5effd640 1 -- 172.21.15.92:0/700826218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d600075d0 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6819f2e0 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.020+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d681ab530 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.021+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6806a850 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.023+0000 7f7d5effd640 1 -- 172.21.15.92:0/700826218 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7d6001c070 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.023+0000 7f7d5effd640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7d540a2c10 0x7f7d540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.024+0000 7f7d6d84d640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7d540a2c10 0x7f7d540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.024+0000 7f7d5effd640 1 -- 172.21.15.92:0/700826218 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7d600cef50 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.024+0000 7f7d6d84d640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7d540a2c10 0x7f7d540a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d640096c0 tx=0x7f7d64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:19.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.027+0000 7f7d5effd640 1 -- 172.21.15.92:0/700826218 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d6009bb00 con 0x7f7d68105ec0 2024-09-11T22:13:19.048 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:18 smithi092 ceph-mon[31173]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-09-11T22:13:19.049 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:18 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/794132792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:13:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.201+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7d681ab760 con 0x7f7d68105ec0 2024-09-11T22:13:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.201+0000 7f7d5effd640 1 -- 172.21.15.92:0/700826218 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7d6009b400 con 0x7f7d68105ec0 2024-09-11T22:13:19.202 INFO:teuthology.orchestra.run.smithi092.stdout:150323855423 2024-09-11T22:13:19.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7d540a2c10 msgr2=0x7f7d540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7d540a2c10 0x7f7d540a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d640096c0 tx=0x7f7d64000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d68105ec0 msgr2=0x7f7d6819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d68105ec0 0x7f7d6819e850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000b970 tx=0x7f7d6000be30 comp rx=0 tx=0).stop 2024-09-11T22:13:19.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 shutdown_connections 2024-09-11T22:13:19.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7d540a2c10 0x7f7d540a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d68105ec0 0x7f7d6819e850 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d681011e0 0x7f7d681a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 --2- 172.21.15.92:0/700826218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d681008c0 0x7f7d681a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 >> 172.21.15.92:0/700826218 conn(0x7f7d680fc7d0 msgr2=0x7f7d6810f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:19.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 shutdown_connections 2024-09-11T22:13:19.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.203+0000 7f7d6fab1640 1 -- 172.21.15.92:0/700826218 wait complete. 2024-09-11T22:13:19.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:18 smithi110 ceph-mon[34830]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-09-11T22:13:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:18 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/794132792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 -- 172.21.15.92:0/2866517847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 msgr2=0x7f020010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 --2- 172.21.15.92:0/2866517847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f020010d950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009cd0 tx=0x7f01f802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 -- 172.21.15.92:0/2866517847 shutdown_connections 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 --2- 172.21.15.92:0/2866517847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f020010d950 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 --2- 172.21.15.92:0/2866517847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0200100a80 0x7f0200100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 --2- 172.21.15.92:0/2866517847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200106a80 0x7f0200106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 -- 172.21.15.92:0/2866517847 >> 172.21.15.92:0/2866517847 conn(0x7f02000fc7a0 msgr2=0x7f02000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 -- 172.21.15.92:0/2866517847 shutdown_connections 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.482+0000 7f0205a30640 1 -- 172.21.15.92:0/2866517847 wait complete. 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 Processor -- start 2024-09-11T22:13:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 -- start start 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0200100a80 0x7f0200199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f0200199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200106a80 0x7f02001a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0200110d50 con 0x7f0200101420 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0200110bd0 con 0x7f0200100a80 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f0205a30640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0200110ed0 con 0x7f0200106a80 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f0200199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f0200199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39746/0 (socket says 172.21.15.92:39746) 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f01f7fff640 1 -- 172.21.15.92:0/3826329350 learned_addr learned my addr 172.21.15.92:0/3826329350 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.483+0000 7f01feffd640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0200100a80 0x7f0200199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f01ff7fe640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200106a80 0x7f02001a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f01f7fff640 1 -- 172.21.15.92:0/3826329350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200106a80 msgr2=0x7f02001a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f01f7fff640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200106a80 0x7f02001a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f01f7fff640 1 -- 172.21.15.92:0/3826329350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0200100a80 msgr2=0x7f0200199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f01f7fff640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0200100a80 0x7f0200199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f01f7fff640 1 -- 172.21.15.92:0/3826329350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02001a4800 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f01f7fff640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f0200199780 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f01e800d950 tx=0x7f01e800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f0204a2e640 1 -- 172.21.15.92:0/3826329350 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8017070 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f0204a2e640 1 -- 172.21.15.92:0/3826329350 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e800de80 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02001a49d0 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f0204a2e640 1 -- 172.21.15.92:0/3826329350 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e800f620 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.484+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02001a4ee0 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.485+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0200100ee0 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.486+0000 7f0204a2e640 1 -- 172.21.15.92:0/3826329350 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f01e8009c30 con 0x7f0200101420 2024-09-11T22:13:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.487+0000 7f0204a2e640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01ec0a2c10 0x7f01ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.487+0000 7f01feffd640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01ec0a2c10 0x7f01ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.487+0000 7f0204a2e640 1 -- 172.21.15.92:0/3826329350 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f01e80cf1a0 con 0x7f0200101420 2024-09-11T22:13:19.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.487+0000 7f01feffd640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01ec0a2c10 0x7f01ec0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009990 tx=0x7f01f0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:19.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.491+0000 7f0204a2e640 1 -- 172.21.15.92:0/3826329350 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01e809bc40 con 0x7f0200101420 2024-09-11T22:13:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.664+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0200109010 con 0x7f0200101420 2024-09-11T22:13:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.664+0000 7f0204a2e640 1 -- 172.21.15.92:0/3826329350 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f01e809b540 con 0x7f0200101420 2024-09-11T22:13:19.665 INFO:teuthology.orchestra.run.smithi092.stdout:335007449115 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01ec0a2c10 msgr2=0x7f01ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01ec0a2c10 0x7f01ec0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009990 tx=0x7f01f0009110 comp rx=0 tx=0).stop 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 msgr2=0x7f0200199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f0200199780 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f01e800d950 tx=0x7f01e800dd50 comp rx=0 tx=0).stop 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 shutdown_connections 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01ec0a2c10 0x7f01ec0a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200106a80 0x7f02001a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0200101420 0x7f0200199780 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 --2- 172.21.15.92:0/3826329350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0200100a80 0x7f0200199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 >> 172.21.15.92:0/3826329350 conn(0x7f02000fc7a0 msgr2=0x7f02000fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 shutdown_connections 2024-09-11T22:13:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:19.667+0000 7f0205a30640 1 -- 172.21.15.92:0/3826329350 wait complete. 2024-09-11T22:13:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:19 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/700826218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:13:20.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:19 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3826329350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:13:20.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855421 got 150323855423 for osd.4 2024-09-11T22:13:20.151 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:19 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/700826218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:13:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:19 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3826329350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:13:20.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:19 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/700826218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:13:20.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:19 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3826329350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:13:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 -- 172.21.15.92:0/851836676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74102d90 msgr2=0x7fdb74103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 --2- 172.21.15.92:0/851836676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74102d90 0x7fdb74103170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009930 tx=0x7fdb6402a420 comp rx=0 tx=0).stop 2024-09-11T22:13:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 -- 172.21.15.92:0/851836676 shutdown_connections 2024-09-11T22:13:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 --2- 172.21.15.92:0/851836676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74108290 0x7fdb7410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 --2- 172.21.15.92:0/851836676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb74103740 0x7fdb74107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 --2- 172.21.15.92:0/851836676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74102d90 0x7fdb74103170 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 -- 172.21.15.92:0/851836676 >> 172.21.15.92:0/851836676 conn(0x7fdb740fc7d0 msgr2=0x7fdb740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 -- 172.21.15.92:0/851836676 shutdown_connections 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.338+0000 7fdb79508640 1 -- 172.21.15.92:0/851836676 wait complete. 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 Processor -- start 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 -- start start 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74102d90 0x7fdb741009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb74103740 0x7fdb74100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74108290 0x7fdb74101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74111f40 con 0x7fdb74108290 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74111dc0 con 0x7fdb74103740 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb79508640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741120c0 con 0x7fdb74102d90 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74108290 0x7fdb74101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74108290 0x7fdb74101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39774/0 (socket says 172.21.15.92:39774) 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 -- 172.21.15.92:0/1535166452 learned_addr learned my addr 172.21.15.92:0/1535166452 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb72ffd640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74102d90 0x7fdb741009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb6bfff640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb74103740 0x7fdb74100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 -- 172.21.15.92:0/1535166452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74102d90 msgr2=0x7fdb741009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74102d90 0x7fdb741009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 -- 172.21.15.92:0/1535166452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb74103740 msgr2=0x7fdb74100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb74103740 0x7fdb74100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb737fe640 1 -- 172.21.15.92:0/1535166452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb741a6960 con 0x7fdb74108290 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.339+0000 7fdb6bfff640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb74103740 0x7fdb74100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.340+0000 7fdb737fe640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74108290 0x7fdb74101470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00eaa0 tx=0x7fdb6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.340+0000 7fdb70ff9640 1 -- 172.21.15.92:0/1535166452 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c00cdb0 con 0x7fdb74108290 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.340+0000 7fdb70ff9640 1 -- 172.21.15.92:0/1535166452 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb6c005020 con 0x7fdb74108290 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.340+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb741a6c50 con 0x7fdb74108290 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.340+0000 7fdb70ff9640 1 -- 172.21.15.92:0/1535166452 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0116d0 con 0x7fdb74108290 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.340+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a7190 con 0x7fdb74108290 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.341+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7406a7f0 con 0x7fdb74108290 2024-09-11T22:13:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.342+0000 7fdb70ff9640 1 -- 172.21.15.92:0/1535166452 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdb6c0051d0 con 0x7fdb74108290 2024-09-11T22:13:20.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.342+0000 7fdb70ff9640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdb600a2c10 0x7fdb600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:20.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.343+0000 7fdb72ffd640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdb600a2c10 0x7fdb600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:20.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.343+0000 7fdb70ff9640 1 -- 172.21.15.92:0/1535166452 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdb6c0cfa30 con 0x7fdb74108290 2024-09-11T22:13:20.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.343+0000 7fdb72ffd640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdb600a2c10 0x7fdb600a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009900 tx=0x7fdb64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:20.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.346+0000 7fdb70ff9640 1 -- 172.21.15.92:0/1535166452 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb6c016030 con 0x7fdb74108290 2024-09-11T22:13:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.517+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdb74103170 con 0x7fdb74108290 2024-09-11T22:13:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.517+0000 7fdb70ff9640 1 -- 172.21.15.92:0/1535166452 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb6c09c4d0 con 0x7fdb74108290 2024-09-11T22:13:20.518 INFO:teuthology.orchestra.run.smithi092.stdout:201863462965 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdb600a2c10 msgr2=0x7fdb600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdb600a2c10 0x7fdb600a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009900 tx=0x7fdb64000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74108290 msgr2=0x7fdb74101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74108290 0x7fdb74101470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00eaa0 tx=0x7fdb6c00ef60 comp rx=0 tx=0).stop 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 shutdown_connections 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdb600a2c10 0x7fdb600a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb74108290 0x7fdb74101470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.519+0000 7fdb79508640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb74103740 0x7fdb74100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.520+0000 7fdb79508640 1 --2- 172.21.15.92:0/1535166452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb74102d90 0x7fdb741009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.520+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 >> 172.21.15.92:0/1535166452 conn(0x7fdb740fc7d0 msgr2=0x7fdb74106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.520+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 shutdown_connections 2024-09-11T22:13:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:20.520+0000 7fdb79508640 1 -- 172.21.15.92:0/1535166452 wait complete. 2024-09-11T22:13:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:20 smithi092 ceph-mon[31173]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-11T22:13:21.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:20 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1535166452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:13:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:20 smithi110 ceph-mon[34830]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-11T22:13:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:20 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1535166452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:13:21.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:20 smithi153 ceph-mon[35045]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-11T22:13:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:20 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1535166452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:13:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 -- 172.21.15.92:0/409895584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90100e20 msgr2=0x7ffa90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 --2- 172.21.15.92:0/409895584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90100e20 0x7ffa90101200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009ca0 tx=0x7ffa8802a950 comp rx=0 tx=0).stop 2024-09-11T22:13:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 -- 172.21.15.92:0/409895584 shutdown_connections 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 --2- 172.21.15.92:0/409895584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa90102ef0 0x7ffa9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 --2- 172.21.15.92:0/409895584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa90102570 0x7ffa901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 --2- 172.21.15.92:0/409895584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90100e20 0x7ffa90101200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 -- 172.21.15.92:0/409895584 >> 172.21.15.92:0/409895584 conn(0x7ffa900fc7d0 msgr2=0x7ffa900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 -- 172.21.15.92:0/409895584 shutdown_connections 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.495+0000 7ffa98466640 1 -- 172.21.15.92:0/409895584 wait complete. 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 Processor -- start 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 -- start start 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa90100e20 0x7ffa901992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90102570 0x7ffa901997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa90102ef0 0x7ffa901a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90110e30 con 0x7ffa90102570 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90110cb0 con 0x7ffa90102ef0 2024-09-11T22:13:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa98466640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90110fb0 con 0x7ffa90100e20 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa95a01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90102570 0x7ffa901997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa95a01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90102570 0x7ffa901997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39788/0 (socket says 172.21.15.92:39788) 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa95a01640 1 -- 172.21.15.92:0/1128296613 learned_addr learned my addr 172.21.15.92:0/1128296613 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa96202640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa90100e20 0x7ffa901992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.496+0000 7ffa96a03640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa90102ef0 0x7ffa901a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa95a01640 1 -- 172.21.15.92:0/1128296613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa90100e20 msgr2=0x7ffa901992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa95a01640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa90100e20 0x7ffa901992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa95a01640 1 -- 172.21.15.92:0/1128296613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa90102ef0 msgr2=0x7ffa901a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa95a01640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa90102ef0 0x7ffa901a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa95a01640 1 -- 172.21.15.92:0/1128296613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa901a4860 con 0x7ffa90102570 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa95a01640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90102570 0x7ffa901997e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400d770 tx=0x7ffa8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa8f7fe640 1 -- 172.21.15.92:0/1128296613 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84017070 con 0x7ffa90102570 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa8f7fe640 1 -- 172.21.15.92:0/1128296613 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa840054d0 con 0x7ffa90102570 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa8f7fe640 1 -- 172.21.15.92:0/1128296613 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84004cf0 con 0x7ffa90102570 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa901a4a30 con 0x7ffa90102570 2024-09-11T22:13:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.497+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa901a4ef0 con 0x7ffa90102570 2024-09-11T22:13:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.498+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa90101200 con 0x7ffa90102570 2024-09-11T22:13:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.499+0000 7ffa8f7fe640 1 -- 172.21.15.92:0/1128296613 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffa84004e90 con 0x7ffa90102570 2024-09-11T22:13:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.500+0000 7ffa8f7fe640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffa780a2c90 0x7ffa780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.500+0000 7ffa96202640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffa780a2c90 0x7ffa780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.500+0000 7ffa8f7fe640 1 -- 172.21.15.92:0/1128296613 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ffa840cee70 con 0x7ffa90102570 2024-09-11T22:13:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.500+0000 7ffa96202640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffa780a2c90 0x7ffa780a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa88004ad0 tx=0x7ffa88004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.504+0000 7ffa8f7fe640 1 -- 172.21.15.92:0/1128296613 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa8409b2c0 con 0x7ffa90102570 2024-09-11T22:13:21.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.676+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ffa90109090 con 0x7ffa90102570 2024-09-11T22:13:21.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.676+0000 7ffa8f7fe640 1 -- 172.21.15.92:0/1128296613 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa8409f5c0 con 0x7ffa90102570 2024-09-11T22:13:21.676 INFO:teuthology.orchestra.run.smithi092.stdout:223338299441 2024-09-11T22:13:21.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffa780a2c90 msgr2=0x7ffa780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffa780a2c90 0x7ffa780a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa88004ad0 tx=0x7ffa88004a60 comp rx=0 tx=0).stop 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90102570 msgr2=0x7ffa901997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90102570 0x7ffa901997e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400d770 tx=0x7ffa8400dc30 comp rx=0 tx=0).stop 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 shutdown_connections 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ffa780a2c90 0x7ffa780a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa90102ef0 0x7ffa901a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90102570 0x7ffa901997e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 --2- 172.21.15.92:0/1128296613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa90100e20 0x7ffa901992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.678+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 >> 172.21.15.92:0/1128296613 conn(0x7ffa900fc7d0 msgr2=0x7ffa900fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.679+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 shutdown_connections 2024-09-11T22:13:21.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:21.679+0000 7ffa98466640 1 -- 172.21.15.92:0/1128296613 wait complete. 2024-09-11T22:13:21.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:21 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1128296613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:13:21.942 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841572 got 283467841573 for osd.9 2024-09-11T22:13:21.942 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:21 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1128296613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:13:22.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:21 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1128296613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:13:22.571 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449113 got 335007449115 for osd.11 2024-09-11T22:13:22.571 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:23.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:22 smithi092 ceph-mon[31173]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s; 7.8 KiB/s, 1 objects/s recovering 2024-09-11T22:13:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:22 smithi110 ceph-mon[34830]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s; 7.8 KiB/s, 1 objects/s recovering 2024-09-11T22:13:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:22 smithi153 ceph-mon[35045]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s; 7.8 KiB/s, 1 objects/s recovering 2024-09-11T22:13:23.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462965 for osd.6 2024-09-11T22:13:23.439 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:23.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 -- 172.21.15.92:0/2343316299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc102ef0 msgr2=0x7f99dc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/2343316299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc102ef0 0x7f99dc10da40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009ca0 tx=0x7f99d402a950 comp rx=0 tx=0).stop 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 -- 172.21.15.92:0/2343316299 shutdown_connections 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/2343316299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc102ef0 0x7f99dc10da40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/2343316299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc102570 0x7f99dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/2343316299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc100e20 0x7f99dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 -- 172.21.15.92:0/2343316299 >> 172.21.15.92:0/2343316299 conn(0x7f99dc0fc7d0 msgr2=0x7f99dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 -- 172.21.15.92:0/2343316299 shutdown_connections 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.602+0000 7f99e2b6e640 1 -- 172.21.15.92:0/2343316299 wait complete. 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 Processor -- start 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 -- start start 2024-09-11T22:13:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc100e20 0x7f99dc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc102570 0x7f99dc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc102ef0 0x7f99dc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110e50 con 0x7f99dc100e20 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110cd0 con 0x7f99dc102570 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e2b6e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110fd0 con 0x7f99dc102ef0 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e090a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc100e20 0x7f99dc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e090a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc100e20 0x7f99dc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39800/0 (socket says 172.21.15.92:39800) 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99e090a640 1 -- 172.21.15.92:0/1239389889 learned_addr learned my addr 172.21.15.92:0/1239389889 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.603+0000 7f99dbfff640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc102570 0x7f99dc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e110b640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc102ef0 0x7f99dc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e090a640 1 -- 172.21.15.92:0/1239389889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc102ef0 msgr2=0x7f99dc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e090a640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc102ef0 0x7f99dc0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e090a640 1 -- 172.21.15.92:0/1239389889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc102570 msgr2=0x7f99dc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e090a640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc102570 0x7f99dc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e090a640 1 -- 172.21.15.92:0/1239389889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc1a68a0 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e090a640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc100e20 0x7f99dc0fcc80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f99d000b4a0 tx=0x7f99d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99d9ffb640 1 -- 172.21.15.92:0/1239389889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0014070 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99d9ffb640 1 -- 172.21.15.92:0/1239389889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99d0010ac0 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99d9ffb640 1 -- 172.21.15.92:0/1239389889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0017c30 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc1a6b90 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.604+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc1a7120 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.605+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc101200 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.606+0000 7f99d9ffb640 1 -- 172.21.15.92:0/1239389889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f99d0017070 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.607+0000 7f99d9ffb640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99cc0a2c90 0x7f99cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.607+0000 7f99dbfff640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99cc0a2c90 0x7f99cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.607+0000 7f99d9ffb640 1 -- 172.21.15.92:0/1239389889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f99d00ceab0 con 0x7f99dc100e20 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.607+0000 7f99dbfff640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99cc0a2c90 0x7f99cc0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0fe710 tx=0x7f99c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.611+0000 7f99d9ffb640 1 -- 172.21.15.92:0/1239389889 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99d009b600 con 0x7f99dc100e20 2024-09-11T22:13:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.784+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f99dc10a0a0 con 0x7f99dc100e20 2024-09-11T22:13:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.785+0000 7f99d9ffb640 1 -- 172.21.15.92:0/1239389889 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99d009af00 con 0x7f99dc100e20 2024-09-11T22:13:23.785 INFO:teuthology.orchestra.run.smithi092.stdout:115964117061 2024-09-11T22:13:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99cc0a2c90 msgr2=0x7f99cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99cc0a2c90 0x7f99cc0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0fe710 tx=0x7f99c800b410 comp rx=0 tx=0).stop 2024-09-11T22:13:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc100e20 msgr2=0x7f99dc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc100e20 0x7f99dc0fcc80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f99d000b4a0 tx=0x7f99d000b960 comp rx=0 tx=0).stop 2024-09-11T22:13:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 shutdown_connections 2024-09-11T22:13:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f99cc0a2c90 0x7f99cc0a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99dc102ef0 0x7f99dc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc102570 0x7f99dc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 --2- 172.21.15.92:0/1239389889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99dc100e20 0x7f99dc0fcc80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 >> 172.21.15.92:0/1239389889 conn(0x7f99dc0fc7d0 msgr2=0x7f99dc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 shutdown_connections 2024-09-11T22:13:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.787+0000 7f99e2b6e640 1 -- 172.21.15.92:0/1239389889 wait complete. 2024-09-11T22:13:23.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 -- 172.21.15.92:0/3650142783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4100a90 msgr2=0x7f26c4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 --2- 172.21.15.92:0/3650142783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4100a90 0x7f26c4100ef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009930 tx=0x7f26ac02a420 comp rx=0 tx=0).stop 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 -- 172.21.15.92:0/3650142783 shutdown_connections 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 --2- 172.21.15.92:0/3650142783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26c4101430 0x7f26c410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 --2- 172.21.15.92:0/3650142783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4100a90 0x7f26c4100ef0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 --2- 172.21.15.92:0/3650142783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4106a90 0x7f26c4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 -- 172.21.15.92:0/3650142783 >> 172.21.15.92:0/3650142783 conn(0x7f26c40fc7d0 msgr2=0x7f26c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 -- 172.21.15.92:0/3650142783 shutdown_connections 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 -- 172.21.15.92:0/3650142783 wait complete. 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 Processor -- start 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.844+0000 7f26ca936640 1 -- start start 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4100a90 0x7f26c4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26c4101430 0x7f26c41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4106a90 0x7f26c419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4111e20 con 0x7f26c4106a90 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4111ca0 con 0x7f26c4100a90 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4111fa0 con 0x7f26c4101430 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4106a90 0x7f26c419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4106a90 0x7f26c419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39822/0 (socket says 172.21.15.92:39822) 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 -- 172.21.15.92:0/633361022 learned_addr learned my addr 172.21.15.92:0/633361022 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c3fff640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4100a90 0x7f26c4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26bbfff640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26c4101430 0x7f26c41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 -- 172.21.15.92:0/633361022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26c4101430 msgr2=0x7f26c41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26c4101430 0x7f26c41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 -- 172.21.15.92:0/633361022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4100a90 msgr2=0x7f26c4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4100a90 0x7f26c4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 -- 172.21.15.92:0/633361022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c419d310 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26bbfff640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26c4101430 0x7f26c41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c3fff640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4100a90 0x7f26c4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c8ed3640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4106a90 0x7f26c419cc30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00cce0 tx=0x7f26bc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c1ffb640 1 -- 172.21.15.92:0/633361022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00f450 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c419d600 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c1ffb640 1 -- 172.21.15.92:0/633361022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26bc00faa0 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26c1ffb640 1 -- 172.21.15.92:0/633361022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc018b10 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.845+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c41a4ee0 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.846+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c4100ef0 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.848+0000 7f26c1ffb640 1 -- 172.21.15.92:0/633361022 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f26bc018cb0 con 0x7f26c4106a90 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.848+0000 7f26c1ffb640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26b00a2c10 0x7f26b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.848+0000 7f26c3fff640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26b00a2c10 0x7f26b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.848+0000 7f26c1ffb640 1 -- 172.21.15.92:0/633361022 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f26bc0cfc20 con 0x7f26c4106a90 2024-09-11T22:13:23.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.849+0000 7f26c3fff640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26b00a2c10 0x7f26b00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26b4005e00 tx=0x7f26b4005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:23.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:23.852+0000 7f26c1ffb640 1 -- 172.21.15.92:0/633361022 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26bc014030 con 0x7f26c4106a90 2024-09-11T22:13:24.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:23 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1239389889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:13:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f26c41996c0 con 0x7f26c4106a90 2024-09-11T22:13:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26c1ffb640 1 -- 172.21.15.92:0/633361022 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f26c41996c0 con 0x7f26c4106a90 2024-09-11T22:13:24.033 INFO:teuthology.orchestra.run.smithi092.stdout:55834574928 2024-09-11T22:13:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26b00a2c10 msgr2=0x7f26b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26b00a2c10 0x7f26b00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26b4005e00 tx=0x7f26b4005c70 comp rx=0 tx=0).stop 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4106a90 msgr2=0x7f26c419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4106a90 0x7f26c419cc30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00cce0 tx=0x7f26bc00a620 comp rx=0 tx=0).stop 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 shutdown_connections 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26b00a2c10 0x7f26b00a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c4106a90 0x7f26c419cc30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26c4101430 0x7f26c41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 --2- 172.21.15.92:0/633361022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4100a90 0x7f26c4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 >> 172.21.15.92:0/633361022 conn(0x7f26c40fc7d0 msgr2=0x7f26c40fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 shutdown_connections 2024-09-11T22:13:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:24.033+0000 7f26ca936640 1 -- 172.21.15.92:0/633361022 wait complete. 2024-09-11T22:13:24.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:23 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1239389889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:13:24.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:23 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1239389889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:13:24.601 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117059 got 115964117061 for osd.3 2024-09-11T22:13:24.601 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:24 smithi110 ceph-mon[34830]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s; 7.8 KiB/s, 1 objects/s recovering 2024-09-11T22:13:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:24 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/633361022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:13:25.247 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:24 smithi092 ceph-mon[31173]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s; 7.8 KiB/s, 1 objects/s recovering 2024-09-11T22:13:25.247 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:24 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/633361022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:13:25.278 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299439 got 223338299441 for osd.7 2024-09-11T22:13:25.278 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:25.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:24 smithi153 ceph-mon[35045]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s; 7.8 KiB/s, 1 objects/s recovering 2024-09-11T22:13:25.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:24 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/633361022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:13:25.734 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574928 for osd.1 2024-09-11T22:13:25.735 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 -- 172.21.15.92:0/2224146267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 msgr2=0x7fd25010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 --2- 172.21.15.92:0/2224146267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd25010ec40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd248009930 tx=0x7fd24802a440 comp rx=0 tx=0).stop 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 -- 172.21.15.92:0/2224146267 shutdown_connections 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 --2- 172.21.15.92:0/2224146267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd25010ec40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 --2- 172.21.15.92:0/2224146267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2501036e0 0x7fd250107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 --2- 172.21.15.92:0/2224146267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd250102d30 0x7fd250103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 -- 172.21.15.92:0/2224146267 >> 172.21.15.92:0/2224146267 conn(0x7fd2500fc7d0 msgr2=0x7fd2500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 -- 172.21.15.92:0/2224146267 shutdown_connections 2024-09-11T22:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.937+0000 7fd2548c6640 1 -- 172.21.15.92:0/2224146267 wait complete. 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 Processor -- start 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 -- start start 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd250102d30 0x7fd250100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2501036e0 0x7fd250100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd250101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd250112010 con 0x7fd2501082a0 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd250111e90 con 0x7fd2501036e0 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd2548c6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd250112190 con 0x7fd250102d30 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd24ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd250101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd24ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd250101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33188/0 (socket says 172.21.15.92:33188) 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd24ed9d640 1 -- 172.21.15.92:0/443637218 learned_addr learned my addr 172.21.15.92:0/443637218 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd24dd9b640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2501036e0 0x7fd250100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.938+0000 7fd24e59c640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd250102d30 0x7fd250100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd24ed9d640 1 -- 172.21.15.92:0/443637218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd250102d30 msgr2=0x7fd250100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd24ed9d640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd250102d30 0x7fd250100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd24ed9d640 1 -- 172.21.15.92:0/443637218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2501036e0 msgr2=0x7fd250100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd24ed9d640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2501036e0 0x7fd250100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd24ed9d640 1 -- 172.21.15.92:0/443637218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2501a6b20 con 0x7fd2501082a0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd24ed9d640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd250101410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd248004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd2477fe640 1 -- 172.21.15.92:0/443637218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24803f030 con 0x7fd2501082a0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2501a6db0 con 0x7fd2501082a0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd2477fe640 1 -- 172.21.15.92:0/443637218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2480035f0 con 0x7fd2501082a0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd2477fe640 1 -- 172.21.15.92:0/443637218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248035070 con 0x7fd2501082a0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.939+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2501a7260 con 0x7fd2501082a0 2024-09-11T22:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.940+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd250103190 con 0x7fd2501082a0 2024-09-11T22:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.941+0000 7fd2477fe640 1 -- 172.21.15.92:0/443637218 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd248003b10 con 0x7fd2501082a0 2024-09-11T22:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.942+0000 7fd2477fe640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd23c0a2c10 0x7fd23c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.942+0000 7fd24e59c640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd23c0a2c10 0x7fd23c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.942+0000 7fd2477fe640 1 -- 172.21.15.92:0/443637218 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd2480f00d0 con 0x7fd2501082a0 2024-09-11T22:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.942+0000 7fd24e59c640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd23c0a2c10 0x7fd23c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd240009f20 tx=0x7fd2400094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:25.946+0000 7fd2477fe640 1 -- 172.21.15.92:0/443637218 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2480bcb70 con 0x7fd2501082a0 2024-09-11T22:13:25.967 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:25.968 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:25.968 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.136+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd25010a2b0 con 0x7fd2501082a0 2024-09-11T22:13:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.136+0000 7fd2477fe640 1 -- 172.21.15.92:0/443637218 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd2480bc470 con 0x7fd2501082a0 2024-09-11T22:13:26.137 INFO:teuthology.orchestra.run.smithi092.stdout:34359738452 2024-09-11T22:13:26.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd23c0a2c10 msgr2=0x7fd23c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:26.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd23c0a2c10 0x7fd23c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd240009f20 tx=0x7fd2400094f0 comp rx=0 tx=0).stop 2024-09-11T22:13:26.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 msgr2=0x7fd250101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:26.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd250101410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd248004270 comp rx=0 tx=0).stop 2024-09-11T22:13:26.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 shutdown_connections 2024-09-11T22:13:26.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd23c0a2c10 0x7fd23c0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:26.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2501082a0 0x7fd250101410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:26.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2501036e0 0x7fd250100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:26.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 --2- 172.21.15.92:0/443637218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd250102d30 0x7fd250100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:26.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.138+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 >> 172.21.15.92:0/443637218 conn(0x7fd2500fc7d0 msgr2=0x7fd250105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:26.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.139+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 shutdown_connections 2024-09-11T22:13:26.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:26.139+0000 7fd2548c6640 1 -- 172.21.15.92:0/443637218 wait complete. 2024-09-11T22:13:26.877 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:26 smithi092 ceph-mon[31173]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 6.2 KiB/s, 1 objects/s recovering 2024-09-11T22:13:26.877 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:26 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/443637218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:13:26.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738452 for osd.0 2024-09-11T22:13:26.906 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:27.091 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:26 smithi153 ceph-mon[35045]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 6.2 KiB/s, 1 objects/s recovering 2024-09-11T22:13:27.091 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:26 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/443637218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:13:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 -- 172.21.15.92:0/1626446086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 msgr2=0x7efe0010bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 --2- 172.21.15.92:0/1626446086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe0010bbf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009900 tx=0x7efdf402a480 comp rx=0 tx=0).stop 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 -- 172.21.15.92:0/1626446086 shutdown_connections 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 --2- 172.21.15.92:0/1626446086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0010c180 0x7efe0010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 --2- 172.21.15.92:0/1626446086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe0010bbf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 --2- 172.21.15.92:0/1626446086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe00102de0 0x7efe001031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 -- 172.21.15.92:0/1626446086 >> 172.21.15.92:0/1626446086 conn(0x7efe000fc7a0 msgr2=0x7efe000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 -- 172.21.15.92:0/1626446086 shutdown_connections 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 -- 172.21.15.92:0/1626446086 wait complete. 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.083+0000 7efe05e71640 1 Processor -- start 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efe05e71640 1 -- start start 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efe05e71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe00102de0 0x7efe00198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efe05e71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe00198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efe05e71640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0010c180 0x7efe0019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efe05e71640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe00111e60 con 0x7efe00103700 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efe05e71640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe00111ce0 con 0x7efe0010c180 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efe05e71640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe00111fe0 con 0x7efe00102de0 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe00198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe00198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33200/0 (socket says 172.21.15.92:33200) 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 -- 172.21.15.92:0/1096936557 learned_addr learned my addr 172.21.15.92:0/1096936557 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdff7fe640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe00102de0 0x7efe00198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfffff640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0010c180 0x7efe0019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 -- 172.21.15.92:0/1096936557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe00102de0 msgr2=0x7efe00198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe00102de0 0x7efe00198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 -- 172.21.15.92:0/1096936557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0010c180 msgr2=0x7efe0019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0010c180 0x7efe0019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdfeffd640 1 -- 172.21.15.92:0/1096936557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0019d2f0 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.084+0000 7efdff7fe640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe00102de0 0x7efe00198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.085+0000 7efdfeffd640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe00198880 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006010 tx=0x7efdf4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.085+0000 7efdfcff9640 1 -- 172.21.15.92:0/1096936557 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf403f030 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.085+0000 7efdfcff9640 1 -- 172.21.15.92:0/1096936557 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf40035f0 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.085+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0019d580 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.085+0000 7efdfcff9640 1 -- 172.21.15.92:0/1096936557 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4035070 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.085+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a4ec0 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.086+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0006afd0 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.087+0000 7efdfcff9640 1 -- 172.21.15.92:0/1096936557 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efdf4003b10 con 0x7efe00103700 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.088+0000 7efdfcff9640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efde80a2c90 0x7efde80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.088+0000 7efdff7fe640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efde80a2c90 0x7efde80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.088+0000 7efdfcff9640 1 -- 172.21.15.92:0/1096936557 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efdf40efca0 con 0x7efe00103700 2024-09-11T22:13:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.088+0000 7efdff7fe640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efde80a2c90 0x7efde80a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efdf000abe0 tx=0x7efdf000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.092+0000 7efdfcff9640 1 -- 172.21.15.92:0/1096936557 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdf40bc740 con 0x7efe00103700 2024-09-11T22:13:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:26 smithi110 ceph-mon[34830]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 6.2 KiB/s, 1 objects/s recovering 2024-09-11T22:13:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:26 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/443637218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:13:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.264+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7efe00106c70 con 0x7efe00103700 2024-09-11T22:13:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.264+0000 7efdfcff9640 1 -- 172.21.15.92:0/1096936557 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efdf40bc040 con 0x7efe00103700 2024-09-11T22:13:27.265 INFO:teuthology.orchestra.run.smithi092.stdout:253403070509 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efde80a2c90 msgr2=0x7efde80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efde80a2c90 0x7efde80a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efdf000abe0 tx=0x7efdf000a5c0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 msgr2=0x7efe00198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe00198880 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006010 tx=0x7efdf4004270 comp rx=0 tx=0).stop 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 shutdown_connections 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efde80a2c90 0x7efde80a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0010c180 0x7efe0019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00103700 0x7efe00198880 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 --2- 172.21.15.92:0/1096936557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe00102de0 0x7efe00198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 >> 172.21.15.92:0/1096936557 conn(0x7efe000fc7a0 msgr2=0x7efe000fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 shutdown_connections 2024-09-11T22:13:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:27.267+0000 7efe05e71640 1 -- 172.21.15.92:0/1096936557 wait complete. 2024-09-11T22:13:28.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:27 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1096936557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:13:28.055 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070509 for osd.8 2024-09-11T22:13:28.055 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:28.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:27 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1096936557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 -- 172.21.15.92:0/844964381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228100e20 msgr2=0x7fc228101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 --2- 172.21.15.92:0/844964381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228100e20 0x7fc228101200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc218009900 tx=0x7fc21802a480 comp rx=0 tx=0).stop 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 -- 172.21.15.92:0/844964381 shutdown_connections 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 --2- 172.21.15.92:0/844964381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228102ef0 0x7fc22810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 --2- 172.21.15.92:0/844964381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc228102570 0x7fc2281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 --2- 172.21.15.92:0/844964381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228100e20 0x7fc228101200 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 -- 172.21.15.92:0/844964381 >> 172.21.15.92:0/844964381 conn(0x7fc2280fc7d0 msgr2=0x7fc2280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 -- 172.21.15.92:0/844964381 shutdown_connections 2024-09-11T22:13:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.219+0000 7fc22cecd640 1 -- 172.21.15.92:0/844964381 wait complete. 2024-09-11T22:13:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 Processor -- start 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 -- start start 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc228100e20 0x7fc228198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228102570 0x7fc228198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228102ef0 0x7fc22819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc228110e30 con 0x7fc228102ef0 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc228110cb0 con 0x7fc228102570 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22cecd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc228110fb0 con 0x7fc228100e20 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc226d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228102ef0 0x7fc22819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228102570 0x7fc228198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc226d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228102ef0 0x7fc22819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33220/0 (socket says 172.21.15.92:33220) 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc226d9d640 1 -- 172.21.15.92:0/1225794849 learned_addr learned my addr 172.21.15.92:0/1225794849 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc22659c640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc228100e20 0x7fc228198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc21ffff640 1 -- 172.21.15.92:0/1225794849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc228100e20 msgr2=0x7fc228198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc21ffff640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc228100e20 0x7fc228198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc21ffff640 1 -- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228102ef0 msgr2=0x7fc22819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc21ffff640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228102ef0 0x7fc22819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.220+0000 7fc21ffff640 1 -- 172.21.15.92:0/1225794849 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22819d3c0 con 0x7fc228102570 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.221+0000 7fc21ffff640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228102570 0x7fc228198950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d720 tx=0x7fc21000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.221+0000 7fc21f7fe640 1 -- 172.21.15.92:0/1225794849 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210017070 con 0x7fc228102570 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.221+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22819d6b0 con 0x7fc228102570 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.221+0000 7fc21f7fe640 1 -- 172.21.15.92:0/1225794849 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc21000de30 con 0x7fc228102570 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.221+0000 7fc21f7fe640 1 -- 172.21.15.92:0/1225794849 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100047a0 con 0x7fc228102570 2024-09-11T22:13:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.221+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281a4f90 con 0x7fc228102570 2024-09-11T22:13:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.222+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228101200 con 0x7fc228102570 2024-09-11T22:13:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.223+0000 7fc21f7fe640 1 -- 172.21.15.92:0/1225794849 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc2100099d0 con 0x7fc228102570 2024-09-11T22:13:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.223+0000 7fc21f7fe640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc2140a2c10 0x7fc2140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.224+0000 7fc22659c640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc2140a2c10 0x7fc2140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.224+0000 7fc21f7fe640 1 -- 172.21.15.92:0/1225794849 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc2100cf470 con 0x7fc228102570 2024-09-11T22:13:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.224+0000 7fc22659c640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc2140a2c10 0x7fc2140a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc218009230 tx=0x7fc2180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.227+0000 7fc21f7fe640 1 -- 172.21.15.92:0/1225794849 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc210020020 con 0x7fc228102570 2024-09-11T22:13:28.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:27 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1096936557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:13:28.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.396+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc228199770 con 0x7fc228102570 2024-09-11T22:13:28.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.396+0000 7fc21f7fe640 1 -- 172.21.15.92:0/1225794849 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc228199770 con 0x7fc228102570 2024-09-11T22:13:28.397 INFO:teuthology.orchestra.run.smithi092.stdout:171798691899 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc2140a2c10 msgr2=0x7fc2140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc2140a2c10 0x7fc2140a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc218009230 tx=0x7fc2180058d0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228102570 msgr2=0x7fc228198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228102570 0x7fc228198950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d720 tx=0x7fc21000dbe0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 shutdown_connections 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc2140a2c10 0x7fc2140a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc228102ef0 0x7fc22819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228102570 0x7fc228198950 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 --2- 172.21.15.92:0/1225794849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc228100e20 0x7fc228198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.398+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 >> 172.21.15.92:0/1225794849 conn(0x7fc2280fc7d0 msgr2=0x7fc22806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:28.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.399+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 shutdown_connections 2024-09-11T22:13:28.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.399+0000 7fc22cecd640 1 -- 172.21.15.92:0/1225794849 wait complete. 2024-09-11T22:13:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 -- 172.21.15.92:0/2353347835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 msgr2=0x7fc208100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 --2- 172.21.15.92:0/2353347835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc208100ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009930 tx=0x7fc1f402a420 comp rx=0 tx=0).stop 2024-09-11T22:13:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 -- 172.21.15.92:0/2353347835 shutdown_connections 2024-09-11T22:13:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 --2- 172.21.15.92:0/2353347835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc208101400 0x7fc20810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 --2- 172.21.15.92:0/2353347835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc208100ec0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 --2- 172.21.15.92:0/2353347835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc208106a60 0x7fc208106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 -- 172.21.15.92:0/2353347835 >> 172.21.15.92:0/2353347835 conn(0x7fc2080fc780 msgr2=0x7fc2080feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 -- 172.21.15.92:0/2353347835 shutdown_connections 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 -- 172.21.15.92:0/2353347835 wait complete. 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 Processor -- start 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.972+0000 7fc20edeb640 1 -- start start 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20edeb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc2080fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20edeb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc208101400 0x7fc2080fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20edeb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc208106a60 0x7fc2080fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20edeb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc208110d10 con 0x7fc208100a60 2024-09-11T22:13:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20edeb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc208110b90 con 0x7fc208101400 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20edeb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc208110e90 con 0x7fc208106a60 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc2080fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc2080fcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33238/0 (socket says 172.21.15.92:33238) 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 -- 172.21.15.92:0/3272087302 learned_addr learned my addr 172.21.15.92:0/3272087302 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc207fff640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc208101400 0x7fc2080fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20d388640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc208106a60 0x7fc2080fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 -- 172.21.15.92:0/3272087302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc208106a60 msgr2=0x7fc2080fd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc208106a60 0x7fc2080fd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 -- 172.21.15.92:0/3272087302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc208101400 msgr2=0x7fc2080fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc208101400 0x7fc2080fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 -- 172.21.15.92:0/3272087302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2081a6850 con 0x7fc208100a60 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20cb87640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc2080fcf50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c890 tx=0x7fc1fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc205ffb640 1 -- 172.21.15.92:0/3272087302 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004220 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc205ffb640 1 -- 172.21.15.92:0/3272087302 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1fc0043c0 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2081a6b40 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.973+0000 7fc205ffb640 1 -- 172.21.15.92:0/3272087302 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004a00 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.974+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2081a7050 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.975+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc208100ec0 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.976+0000 7fc205ffb640 1 -- 172.21.15.92:0/3272087302 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc1fc004ba0 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.976+0000 7fc205ffb640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.976+0000 7fc207fff640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.976+0000 7fc205ffb640 1 -- 172.21.15.92:0/3272087302 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc1fc0cf1c0 con 0x7fc208100a60 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.977+0000 7fc207fff640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009900 tx=0x7fc1f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:28.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:28.980+0000 7fc205ffb640 1 -- 172.21.15.92:0/3272087302 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1fc014030 con 0x7fc208100a60 2024-09-11T22:13:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.154+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc2080fe480 con 0x7fc208100a60 2024-09-11T22:13:29.155 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:28 smithi092 ceph-mon[31173]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-09-11T22:13:29.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:28 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1225794849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:13:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.154+0000 7fc205ffb640 1 -- 172.21.15.92:0/3272087302 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc1fc09bc60 con 0x7fc208100a60 2024-09-11T22:13:29.156 INFO:teuthology.orchestra.run.smithi092.stdout:77309411403 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.156+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc1f80a2c10 msgr2=0x7fc1f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.156+0000 7fc20edeb640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009900 tx=0x7fc1f4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 msgr2=0x7fc2080fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc2080fcf50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c890 tx=0x7fc1fc00cd50 comp rx=0 tx=0).stop 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 shutdown_connections 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc208106a60 0x7fc2080fd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc208101400 0x7fc2080fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 --2- 172.21.15.92:0/3272087302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc208100a60 0x7fc2080fcf50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 >> 172.21.15.92:0/3272087302 conn(0x7fc2080fc780 msgr2=0x7fc20806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 shutdown_connections 2024-09-11T22:13:29.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:29.157+0000 7fc20edeb640 1 -- 172.21.15.92:0/3272087302 wait complete. 2024-09-11T22:13:29.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:28 smithi110 ceph-mon[34830]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-09-11T22:13:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:28 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1225794849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:13:29.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:28 smithi153 ceph-mon[35045]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-09-11T22:13:29.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:28 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1225794849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:13:29.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691897 got 171798691899 for osd.5 2024-09-11T22:13:29.490 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:29 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3272087302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:13:30.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:29 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3272087302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:13:30.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:29 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3272087302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:13:30.449 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411403 for osd.2 2024-09-11T22:13:30.449 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:30.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 -- 172.21.15.92:0/1370684357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 msgr2=0x7f37fc100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:30.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 --2- 172.21.15.92:0/1370684357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc100f10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802a470 comp rx=0 tx=0).stop 2024-09-11T22:13:30.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 -- 172.21.15.92:0/1370684357 shutdown_connections 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 --2- 172.21.15.92:0/1370684357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37fc105f40 0x7f37fc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 --2- 172.21.15.92:0/1370684357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37fc101450 0x7f37fc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 --2- 172.21.15.92:0/1370684357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc100f10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 -- 172.21.15.92:0/1370684357 >> 172.21.15.92:0/1370684357 conn(0x7f37fc0fc7a0 msgr2=0x7f37fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.625+0000 7f380378c640 1 -- 172.21.15.92:0/1370684357 shutdown_connections 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.626+0000 7f380378c640 1 -- 172.21.15.92:0/1370684357 wait complete. 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.626+0000 7f380378c640 1 Processor -- start 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.626+0000 7f380378c640 1 -- start start 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37fc101450 0x7f37fc1a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37fc105f40 0x7f37fc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc118470 con 0x7f37fc100b30 2024-09-11T22:13:30.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1182f0 con 0x7f37fc105f40 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1185f0 con 0x7f37fc101450 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33266/0 (socket says 172.21.15.92:33266) 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 -- 172.21.15.92:0/3575656947 learned_addr learned my addr 172.21.15.92:0/3575656947 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3800d27640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37fc101450 0x7f37fc1a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801d29640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37fc105f40 0x7f37fc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 -- 172.21.15.92:0/3575656947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37fc101450 msgr2=0x7f37fc1a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37fc101450 0x7f37fc1a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 -- 172.21.15.92:0/3575656947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37fc105f40 msgr2=0x7f37fc19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37fc105f40 0x7f37fc19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 -- 172.21.15.92:0/3575656947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc19f050 con 0x7f37fc100b30 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3800d27640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37fc101450 0x7f37fc1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f3801528640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc1a4760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f37f8006010 tx=0x7f37f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f37f27fc640 1 -- 172.21.15.92:0/3575656947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f803f030 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f37f27fc640 1 -- 172.21.15.92:0/3575656947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37f80035f0 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37fc19f2e0 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f37f27fc640 1 -- 172.21.15.92:0/3575656947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8035070 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.627+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37fc1ab630 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.628+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37fc105920 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.630+0000 7f37f27fc640 1 -- 172.21.15.92:0/3575656947 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f37f8003b10 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.630+0000 7f37f27fc640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f37e80a2c10 0x7f37e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.630+0000 7f3800d27640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f37e80a2c10 0x7f37e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.630+0000 7f37f27fc640 1 -- 172.21.15.92:0/3575656947 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f37f80f0370 con 0x7f37fc100b30 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.631+0000 7f3800d27640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f37e80a2c10 0x7f37e80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37ec006fd0 tx=0x7f37ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:30.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.633+0000 7f37f27fc640 1 -- 172.21.15.92:0/3575656947 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37f80bce10 con 0x7f37fc100b30 2024-09-11T22:13:30.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.806+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f37fc1ab800 con 0x7f37fc100b30 2024-09-11T22:13:30.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.807+0000 7f37f27fc640 1 -- 172.21.15.92:0/3575656947 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f37f80bc710 con 0x7f37fc100b30 2024-09-11T22:13:30.808 INFO:teuthology.orchestra.run.smithi092.stdout:304942678050 2024-09-11T22:13:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f37e80a2c10 msgr2=0x7f37e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f37e80a2c10 0x7f37e80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37ec006fd0 tx=0x7f37ec008040 comp rx=0 tx=0).stop 2024-09-11T22:13:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 msgr2=0x7f37fc1a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc1a4760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f37f8006010 tx=0x7f37f8004270 comp rx=0 tx=0).stop 2024-09-11T22:13:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 shutdown_connections 2024-09-11T22:13:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f37e80a2c10 0x7f37e80a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37fc105f40 0x7f37fc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37fc101450 0x7f37fc1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 --2- 172.21.15.92:0/3575656947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37fc100b30 0x7f37fc1a4760 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 >> 172.21.15.92:0/3575656947 conn(0x7f37fc0fc7a0 msgr2=0x7f37fc10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 shutdown_connections 2024-09-11T22:13:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:30.809+0000 7f380378c640 1 -- 172.21.15.92:0/3575656947 wait complete. 2024-09-11T22:13:31.168 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:30 smithi092 ceph-mon[31173]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-09-11T22:13:31.169 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:30 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3575656947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:13:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:30 smithi110 ceph-mon[34830]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-09-11T22:13:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:30 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3575656947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:13:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:30 smithi153 ceph-mon[35045]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-09-11T22:13:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:30 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3575656947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:13:31.497 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678047 got 304942678050 for osd.10 2024-09-11T22:13:31.497 DEBUG:teuthology.parallel:result is None 2024-09-11T22:13:31.498 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:13:31.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:13:31.747 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 -- 172.21.15.92:0/3838436944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c076720 msgr2=0x7f3d0c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 --2- 172.21.15.92:0/3838436944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c076720 0x7f3d0c076b00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009900 tx=0x7f3cfc02a3f0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 -- 172.21.15.92:0/3838436944 shutdown_connections 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 --2- 172.21.15.92:0/3838436944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c112a30 0x7f3d0c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 --2- 172.21.15.92:0/3838436944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c077040 0x7f3d0c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 --2- 172.21.15.92:0/3838436944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c076720 0x7f3d0c076b00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 -- 172.21.15.92:0/3838436944 >> 172.21.15.92:0/3838436944 conn(0x7f3d0c0ffa90 msgr2=0x7f3d0c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 -- 172.21.15.92:0/3838436944 shutdown_connections 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.251+0000 7f3d1086b640 1 -- 172.21.15.92:0/3838436944 wait complete. 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 Processor -- start 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 -- start start 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c076720 0x7f3d0c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c077040 0x7f3d0c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c112a30 0x7f3d0c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1184c0 con 0x7f3d0c112a30 2024-09-11T22:13:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c118340 con 0x7f3d0c076720 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d1086b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c118640 con 0x7f3d0c077040 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d0ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c112a30 0x7f3d0c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d0ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c112a30 0x7f3d0c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33298/0 (socket says 172.21.15.92:33298) 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.252+0000 7f3d0ad9d640 1 -- 172.21.15.92:0/1041057485 learned_addr learned my addr 172.21.15.92:0/1041057485 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d0a59c640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c076720 0x7f3d0c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d0ad9d640 1 -- 172.21.15.92:0/1041057485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c077040 msgr2=0x7f3d0c1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d0ad9d640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c077040 0x7f3d0c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d0ad9d640 1 -- 172.21.15.92:0/1041057485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c076720 msgr2=0x7f3d0c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d0ad9d640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c076720 0x7f3d0c1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d0ad9d640 1 -- 172.21.15.92:0/1041057485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c19efd0 con 0x7f3d0c112a30 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d0ad9d640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c112a30 0x7f3d0c19e830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009fd0 tx=0x7f3d0400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d037fe640 1 -- 172.21.15.92:0/1041057485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04004d20 con 0x7f3d0c112a30 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d037fe640 1 -- 172.21.15.92:0/1041057485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d04005020 con 0x7f3d0c112a30 2024-09-11T22:13:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c19f2c0 con 0x7f3d0c112a30 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d037fe640 1 -- 172.21.15.92:0/1041057485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040116b0 con 0x7f3d0c112a30 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.253+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1ab510 con 0x7f3d0c112a30 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.254+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c077e60 con 0x7f3d0c112a30 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.256+0000 7f3d037fe640 1 -- 172.21.15.92:0/1041057485 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3d040051d0 con 0x7f3d0c112a30 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.256+0000 7f3d037fe640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.257+0000 7f3d0a59c640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.257+0000 7f3d037fe640 1 -- 172.21.15.92:0/1041057485 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3d040cfc20 con 0x7f3d0c112a30 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.257+0000 7f3d0a59c640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc006010 tx=0x7f3cfc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.260+0000 7f3d037fe640 1 -- 172.21.15.92:0/1041057485 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d04016030 con 0x7f3d0c112a30 2024-09-11T22:13:32.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.424+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d0c062630 con 0x7f3cf80a2c10 2024-09-11T22:13:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.434+0000 7f3d037fe640 1 -- 172.21.15.92:0/1041057485 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340731 (secure 0 0 0) 0x7f3d0c062630 con 0x7f3cf80a2c10 2024-09-11T22:13:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:32.441 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T22:13:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3cf80a2c10 msgr2=0x7f3cf80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc006010 tx=0x7f3cfc000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c112a30 msgr2=0x7f3d0c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c112a30 0x7f3d0c19e830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009fd0 tx=0x7f3d0400ee60 comp rx=0 tx=0).stop 2024-09-11T22:13:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 shutdown_connections 2024-09-11T22:13:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d0c112a30 0x7f3d0c19e830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d0c077040 0x7f3d0c1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 --2- 172.21.15.92:0/1041057485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c076720 0x7f3d0c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 >> 172.21.15.92:0/1041057485 conn(0x7f3d0c0ffa90 msgr2=0x7f3d0c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 shutdown_connections 2024-09-11T22:13:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:32.438+0000 7f3d1086b640 1 -- 172.21.15.92:0/1041057485 wait complete. 2024-09-11T22:13:33.096 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:32 smithi092 ceph-mon[31173]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-09-11T22:13:33.124 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-11T22:13:31.704083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661294,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":651,"num_read_kb":450,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1244242,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382416,"kb_used_data":12300,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559408,"statfs":{"total":1151940427776,"available":1151548833792,"internally_reserved":0,"allocated":12595200,"data_stored":8076080,"data_compressed":406841,"data_compressed_allocated":548864,"data_compressed_original":2633728,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":63506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"2.1f","version":"90'13","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:27.218406+0000","last_change":"2024-09-11T22:12:00.751331+0000","last_active":"2024-09-11T22:13:27.218406+0000","last_peered":"2024-09-11T22:13:27.218406+0000","last_clean":"2024-09-11T22:13:27.218406+0000","last_became_active":"2024-09-11T22:12:00.751034+0000","last_became_peered":"2024-09-11T22:12:00.751034+0000","last_unstale":"2024-09-11T22:13:27.218406+0000","last_undegraded":"2024-09-11T22:13:27.218406+0000","last_fullsized":"2024-09-11T22:13:27.218406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:14.418754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786549+0000","last_change":"2024-09-11T22:13:16.786159+0000","last_active":"2024-09-11T22:13:16.786549+0000","last_peered":"2024-09-11T22:13:16.786549+0000","last_clean":"2024-09-11T22:13:16.786549+0000","last_became_active":"2024-09-11T22:13:16.785880+0000","last_became_peered":"2024-09-11T22:13:16.785880+0000","last_unstale":"2024-09-11T22:13:16.786549+0000","last_undegraded":"2024-09-11T22:13:16.786549+0000","last_fullsized":"2024-09-11T22:13:16.786549+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:54.511343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780480+0000","last_change":"2024-09-11T22:12:09.818054+0000","last_active":"2024-09-11T22:13:16.780480+0000","last_peered":"2024-09-11T22:13:16.780480+0000","last_clean":"2024-09-11T22:13:16.780480+0000","last_became_active":"2024-09-11T22:12:09.817527+0000","last_became_peered":"2024-09-11T22:12:09.817527+0000","last_unstale":"2024-09-11T22:13:16.780480+0000","last_undegraded":"2024-09-11T22:13:16.780480+0000","last_fullsized":"2024-09-11T22:13:16.780480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:26.496214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038623+0000","last_change":"2024-09-11T22:12:00.817451+0000","last_active":"2024-09-11T22:13:17.038623+0000","last_peered":"2024-09-11T22:13:17.038623+0000","last_clean":"2024-09-11T22:13:17.038623+0000","last_became_active":"2024-09-11T22:12:00.817047+0000","last_became_peered":"2024-09-11T22:12:00.817047+0000","last_unstale":"2024-09-11T22:13:17.038623+0000","last_undegraded":"2024-09-11T22:13:17.038623+0000","last_fullsized":"2024-09-11T22:13:17.038623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:48.450520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778306+0000","last_change":"2024-09-11T22:12:02.764638+0000","last_active":"2024-09-11T22:13:16.778306+0000","last_peered":"2024-09-11T22:13:16.778306+0000","last_clean":"2024-09-11T22:13:16.778306+0000","last_became_active":"2024-09-11T22:12:02.764329+0000","last_became_peered":"2024-09-11T22:12:02.764329+0000","last_unstale":"2024-09-11T22:13:16.778306+0000","last_undegraded":"2024-09-11T22:13:16.778306+0000","last_fullsized":"2024-09-11T22:13:16.778306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:27.322976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776076+0000","last_change":"2024-09-11T22:12:09.817478+0000","last_active":"2024-09-11T22:13:16.776076+0000","last_peered":"2024-09-11T22:13:16.776076+0000","last_clean":"2024-09-11T22:13:16.776076+0000","last_became_active":"2024-09-11T22:12:09.817098+0000","last_became_peered":"2024-09-11T22:12:09.817098+0000","last_unstale":"2024-09-11T22:13:16.776076+0000","last_undegraded":"2024-09-11T22:13:16.776076+0000","last_fullsized":"2024-09-11T22:13:16.776076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:50.066572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776891+0000","last_change":"2024-09-11T22:12:00.754280+0000","last_active":"2024-09-11T22:13:16.776891+0000","last_peered":"2024-09-11T22:13:16.776891+0000","last_clean":"2024-09-11T22:13:16.776891+0000","last_became_active":"2024-09-11T22:12:00.753906+0000","last_became_peered":"2024-09-11T22:12:00.753906+0000","last_unstale":"2024-09-11T22:13:16.776891+0000","last_undegraded":"2024-09-11T22:13:16.776891+0000","last_fullsized":"2024-09-11T22:13:16.776891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:01.049033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784841+0000","last_change":"2024-09-11T22:12:02.765037+0000","last_active":"2024-09-11T22:13:16.784841+0000","last_peered":"2024-09-11T22:13:16.784841+0000","last_clean":"2024-09-11T22:13:16.784841+0000","last_became_active":"2024-09-11T22:12:02.764730+0000","last_became_peered":"2024-09-11T22:12:02.764730+0000","last_unstale":"2024-09-11T22:13:16.784841+0000","last_undegraded":"2024-09-11T22:13:16.784841+0000","last_fullsized":"2024-09-11T22:13:16.784841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:13.974389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776396+0000","last_change":"2024-09-11T22:12:09.817801+0000","last_active":"2024-09-11T22:13:16.776396+0000","last_peered":"2024-09-11T22:13:16.776396+0000","last_clean":"2024-09-11T22:13:16.776396+0000","last_became_active":"2024-09-11T22:12:09.817389+0000","last_became_peered":"2024-09-11T22:12:09.817389+0000","last_unstale":"2024-09-11T22:13:16.776396+0000","last_undegraded":"2024-09-11T22:13:16.776396+0000","last_fullsized":"2024-09-11T22:13:16.776396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:55.152113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776816+0000","last_change":"2024-09-11T22:12:00.752216+0000","last_active":"2024-09-11T22:13:16.776816+0000","last_peered":"2024-09-11T22:13:16.776816+0000","last_clean":"2024-09-11T22:13:16.776816+0000","last_became_active":"2024-09-11T22:12:00.751641+0000","last_became_peered":"2024-09-11T22:12:00.751641+0000","last_unstale":"2024-09-11T22:13:16.776816+0000","last_undegraded":"2024-09-11T22:13:16.776816+0000","last_fullsized":"2024-09-11T22:13:16.776816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784299+0000","last_change":"2024-09-11T22:12:02.776766+0000","last_active":"2024-09-11T22:13:16.784299+0000","last_peered":"2024-09-11T22:13:16.784299+0000","last_clean":"2024-09-11T22:13:16.784299+0000","last_became_active":"2024-09-11T22:12:02.775985+0000","last_became_peered":"2024-09-11T22:12:02.775985+0000","last_unstale":"2024-09-11T22:13:16.784299+0000","last_undegraded":"2024-09-11T22:13:16.784299+0000","last_fullsized":"2024-09-11T22:13:16.784299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:39.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776177+0000","last_change":"2024-09-11T22:12:09.817906+0000","last_active":"2024-09-11T22:13:16.776177+0000","last_peered":"2024-09-11T22:13:16.776177+0000","last_clean":"2024-09-11T22:13:16.776177+0000","last_became_active":"2024-09-11T22:12:09.817404+0000","last_became_peered":"2024-09-11T22:12:09.817404+0000","last_unstale":"2024-09-11T22:13:16.776177+0000","last_undegraded":"2024-09-11T22:13:16.776177+0000","last_fullsized":"2024-09-11T22:13:16.776177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:17:29.957890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779692+0000","last_change":"2024-09-11T22:12:02.770932+0000","last_active":"2024-09-11T22:13:16.779692+0000","last_peered":"2024-09-11T22:13:16.779692+0000","last_clean":"2024-09-11T22:13:16.779692+0000","last_became_active":"2024-09-11T22:12:02.770619+0000","last_became_peered":"2024-09-11T22:12:02.770619+0000","last_unstale":"2024-09-11T22:13:16.779692+0000","last_undegraded":"2024-09-11T22:13:16.779692+0000","last_fullsized":"2024-09-11T22:13:16.779692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:31.746892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.791862+0000","last_change":"2024-09-11T22:13:16.791862+0000","last_active":"2024-09-11T22:13:16.791862+0000","last_peered":"2024-09-11T22:13:16.791862+0000","last_clean":"2024-09-11T22:13:16.791862+0000","last_became_active":"2024-09-11T22:13:16.787780+0000","last_became_peered":"2024-09-11T22:13:16.787780+0000","last_unstale":"2024-09-11T22:13:16.791862+0000","last_undegraded":"2024-09-11T22:13:16.791862+0000","last_fullsized":"2024-09-11T22:13:16.791862+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.519704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776411+0000","last_change":"2024-09-11T22:12:10.199589+0000","last_active":"2024-09-11T22:13:16.776411+0000","last_peered":"2024-09-11T22:13:16.776411+0000","last_clean":"2024-09-11T22:13:16.776411+0000","last_became_active":"2024-09-11T22:12:10.199185+0000","last_became_peered":"2024-09-11T22:12:10.199185+0000","last_unstale":"2024-09-11T22:13:16.776411+0000","last_undegraded":"2024-09-11T22:13:16.776411+0000","last_fullsized":"2024-09-11T22:13:16.776411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:51.081016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037739+0000","last_change":"2024-09-11T22:12:02.763355+0000","last_active":"2024-09-11T22:13:17.037739+0000","last_peered":"2024-09-11T22:13:17.037739+0000","last_clean":"2024-09-11T22:13:17.037739+0000","last_became_active":"2024-09-11T22:12:02.763060+0000","last_became_peered":"2024-09-11T22:12:02.763060+0000","last_unstale":"2024-09-11T22:13:17.037739+0000","last_undegraded":"2024-09-11T22:13:17.037739+0000","last_fullsized":"2024-09-11T22:13:17.037739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:22.776503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780037+0000","last_change":"2024-09-11T22:12:00.754158+0000","last_active":"2024-09-11T22:13:16.780037+0000","last_peered":"2024-09-11T22:13:16.780037+0000","last_clean":"2024-09-11T22:13:16.780037+0000","last_became_active":"2024-09-11T22:12:00.753706+0000","last_became_peered":"2024-09-11T22:12:00.753706+0000","last_unstale":"2024-09-11T22:13:16.780037+0000","last_undegraded":"2024-09-11T22:13:16.780037+0000","last_fullsized":"2024-09-11T22:13:16.780037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:40.470704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776546+0000","last_change":"2024-09-11T22:12:09.810374+0000","last_active":"2024-09-11T22:13:16.776546+0000","last_peered":"2024-09-11T22:13:16.776546+0000","last_clean":"2024-09-11T22:13:16.776546+0000","last_became_active":"2024-09-11T22:12:09.809915+0000","last_became_peered":"2024-09-11T22:12:09.809915+0000","last_unstale":"2024-09-11T22:13:16.776546+0000","last_undegraded":"2024-09-11T22:13:16.776546+0000","last_fullsized":"2024-09-11T22:13:16.776546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:25.554920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778581+0000","last_change":"2024-09-11T22:12:02.769350+0000","last_active":"2024-09-11T22:13:16.778581+0000","last_peered":"2024-09-11T22:13:16.778581+0000","last_clean":"2024-09-11T22:13:16.778581+0000","last_became_active":"2024-09-11T22:12:02.768896+0000","last_became_peered":"2024-09-11T22:12:02.768896+0000","last_unstale":"2024-09-11T22:13:16.778581+0000","last_undegraded":"2024-09-11T22:13:16.778581+0000","last_fullsized":"2024-09-11T22:13:16.778581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:12.490938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037624+0000","last_change":"2024-09-11T22:12:00.743281+0000","last_active":"2024-09-11T22:13:17.037624+0000","last_peered":"2024-09-11T22:13:17.037624+0000","last_clean":"2024-09-11T22:13:17.037624+0000","last_became_active":"2024-09-11T22:12:00.742977+0000","last_became_peered":"2024-09-11T22:12:00.742977+0000","last_unstale":"2024-09-11T22:13:17.037624+0000","last_undegraded":"2024-09-11T22:13:17.037624+0000","last_fullsized":"2024-09-11T22:13:17.037624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:28.890328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775712+0000","last_change":"2024-09-11T22:12:09.815471+0000","last_active":"2024-09-11T22:13:16.775712+0000","last_peered":"2024-09-11T22:13:16.775712+0000","last_clean":"2024-09-11T22:13:16.775712+0000","last_became_active":"2024-09-11T22:12:09.815098+0000","last_became_peered":"2024-09-11T22:12:09.815098+0000","last_unstale":"2024-09-11T22:13:16.775712+0000","last_undegraded":"2024-09-11T22:13:16.775712+0000","last_fullsized":"2024-09-11T22:13:16.775712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:36.375848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776142+0000","last_change":"2024-09-11T22:12:02.763597+0000","last_active":"2024-09-11T22:13:16.776142+0000","last_peered":"2024-09-11T22:13:16.776142+0000","last_clean":"2024-09-11T22:13:16.776142+0000","last_became_active":"2024-09-11T22:12:02.763242+0000","last_became_peered":"2024-09-11T22:12:02.763242+0000","last_unstale":"2024-09-11T22:13:16.776142+0000","last_undegraded":"2024-09-11T22:13:16.776142+0000","last_fullsized":"2024-09-11T22:13:16.776142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:24.968799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.785828+0000","last_change":"2024-09-11T22:12:00.817604+0000","last_active":"2024-09-11T22:13:16.785828+0000","last_peered":"2024-09-11T22:13:16.785828+0000","last_clean":"2024-09-11T22:13:16.785828+0000","last_became_active":"2024-09-11T22:12:00.817377+0000","last_became_peered":"2024-09-11T22:12:00.817377+0000","last_unstale":"2024-09-11T22:13:16.785828+0000","last_undegraded":"2024-09-11T22:13:16.785828+0000","last_fullsized":"2024-09-11T22:13:16.785828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:27.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781909+0000","last_change":"2024-09-11T22:13:16.781909+0000","last_active":"2024-09-11T22:13:16.781909+0000","last_peered":"2024-09-11T22:13:16.781909+0000","last_clean":"2024-09-11T22:13:16.781909+0000","last_became_active":"2024-09-11T22:13:16.781242+0000","last_became_peered":"2024-09-11T22:13:16.781242+0000","last_unstale":"2024-09-11T22:13:16.781909+0000","last_undegraded":"2024-09-11T22:13:16.781909+0000","last_fullsized":"2024-09-11T22:13:16.781909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:23.724541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777977+0000","last_change":"2024-09-11T22:12:02.773380+0000","last_active":"2024-09-11T22:13:16.777977+0000","last_peered":"2024-09-11T22:13:16.777977+0000","last_clean":"2024-09-11T22:13:16.777977+0000","last_became_active":"2024-09-11T22:12:02.773076+0000","last_became_peered":"2024-09-11T22:12:02.773076+0000","last_unstale":"2024-09-11T22:13:16.777977+0000","last_undegraded":"2024-09-11T22:13:16.777977+0000","last_fullsized":"2024-09-11T22:13:16.777977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:35.475581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777306+0000","last_change":"2024-09-11T22:12:00.746438+0000","last_active":"2024-09-11T22:13:16.777306+0000","last_peered":"2024-09-11T22:13:16.777306+0000","last_clean":"2024-09-11T22:13:16.777306+0000","last_became_active":"2024-09-11T22:12:00.746034+0000","last_became_peered":"2024-09-11T22:12:00.746034+0000","last_unstale":"2024-09-11T22:13:16.777306+0000","last_undegraded":"2024-09-11T22:13:16.777306+0000","last_fullsized":"2024-09-11T22:13:16.777306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:49.156748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780666+0000","last_change":"2024-09-11T22:12:09.812553+0000","last_active":"2024-09-11T22:13:16.780666+0000","last_peered":"2024-09-11T22:13:16.780666+0000","last_clean":"2024-09-11T22:13:16.780666+0000","last_became_active":"2024-09-11T22:12:09.812092+0000","last_became_peered":"2024-09-11T22:12:09.812092+0000","last_unstale":"2024-09-11T22:13:16.780666+0000","last_undegraded":"2024-09-11T22:13:16.780666+0000","last_fullsized":"2024-09-11T22:13:16.780666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:13.947799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038274+0000","last_change":"2024-09-11T22:12:02.763399+0000","last_active":"2024-09-11T22:13:17.038274+0000","last_peered":"2024-09-11T22:13:17.038274+0000","last_clean":"2024-09-11T22:13:17.038274+0000","last_became_active":"2024-09-11T22:12:02.763082+0000","last_became_peered":"2024-09-11T22:12:02.763082+0000","last_unstale":"2024-09-11T22:13:17.038274+0000","last_undegraded":"2024-09-11T22:13:17.038274+0000","last_fullsized":"2024-09-11T22:13:17.038274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:52.267960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775942+0000","last_change":"2024-09-11T22:12:00.751318+0000","last_active":"2024-09-11T22:13:16.775942+0000","last_peered":"2024-09-11T22:13:16.775942+0000","last_clean":"2024-09-11T22:13:16.775942+0000","last_became_active":"2024-09-11T22:12:00.751022+0000","last_became_peered":"2024-09-11T22:12:00.751022+0000","last_unstale":"2024-09-11T22:13:16.775942+0000","last_undegraded":"2024-09-11T22:13:16.775942+0000","last_fullsized":"2024-09-11T22:13:16.775942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:23:26.163502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037272+0000","last_change":"2024-09-11T22:12:10.200279+0000","last_active":"2024-09-11T22:13:17.037272+0000","last_peered":"2024-09-11T22:13:17.037272+0000","last_clean":"2024-09-11T22:13:17.037272+0000","last_became_active":"2024-09-11T22:12:10.199920+0000","last_became_peered":"2024-09-11T22:12:10.199920+0000","last_unstale":"2024-09-11T22:13:17.037272+0000","last_undegraded":"2024-09-11T22:13:17.037272+0000","last_fullsized":"2024-09-11T22:13:17.037272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:59.012735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037898+0000","last_change":"2024-09-11T22:12:02.755565+0000","last_active":"2024-09-11T22:13:17.037898+0000","last_peered":"2024-09-11T22:13:17.037898+0000","last_clean":"2024-09-11T22:13:17.037898+0000","last_became_active":"2024-09-11T22:12:02.755236+0000","last_became_peered":"2024-09-11T22:12:02.755236+0000","last_unstale":"2024-09-11T22:13:17.037898+0000","last_undegraded":"2024-09-11T22:13:17.037898+0000","last_fullsized":"2024-09-11T22:13:17.037898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.588976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776929+0000","last_change":"2024-09-11T22:12:00.818634+0000","last_active":"2024-09-11T22:13:16.776929+0000","last_peered":"2024-09-11T22:13:16.776929+0000","last_clean":"2024-09-11T22:13:16.776929+0000","last_became_active":"2024-09-11T22:12:00.817917+0000","last_became_peered":"2024-09-11T22:12:00.817917+0000","last_unstale":"2024-09-11T22:13:16.776929+0000","last_undegraded":"2024-09-11T22:13:16.776929+0000","last_fullsized":"2024-09-11T22:13:16.776929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:42.382962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778793+0000","last_change":"2024-09-11T22:12:09.814770+0000","last_active":"2024-09-11T22:13:16.778793+0000","last_peered":"2024-09-11T22:13:16.778793+0000","last_clean":"2024-09-11T22:13:16.778793+0000","last_became_active":"2024-09-11T22:12:09.814322+0000","last_became_peered":"2024-09-11T22:12:09.814322+0000","last_unstale":"2024-09-11T22:13:16.778793+0000","last_undegraded":"2024-09-11T22:13:16.778793+0000","last_fullsized":"2024-09-11T22:13:16.778793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.666161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781247+0000","last_change":"2024-09-11T22:12:02.762939+0000","last_active":"2024-09-11T22:13:16.781247+0000","last_peered":"2024-09-11T22:13:16.781247+0000","last_clean":"2024-09-11T22:13:16.781247+0000","last_became_active":"2024-09-11T22:12:02.762579+0000","last_became_peered":"2024-09-11T22:12:02.762579+0000","last_unstale":"2024-09-11T22:13:16.781247+0000","last_undegraded":"2024-09-11T22:13:16.781247+0000","last_fullsized":"2024-09-11T22:13:16.781247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:35.124612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775930+0000","last_change":"2024-09-11T22:12:00.818094+0000","last_active":"2024-09-11T22:13:16.775930+0000","last_peered":"2024-09-11T22:13:16.775930+0000","last_clean":"2024-09-11T22:13:16.775930+0000","last_became_active":"2024-09-11T22:12:00.817723+0000","last_became_peered":"2024-09-11T22:12:00.817723+0000","last_unstale":"2024-09-11T22:13:16.775930+0000","last_undegraded":"2024-09-11T22:13:16.775930+0000","last_fullsized":"2024-09-11T22:13:16.775930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:32.618689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776157+0000","last_change":"2024-09-11T22:12:10.199778+0000","last_active":"2024-09-11T22:13:16.776157+0000","last_peered":"2024-09-11T22:13:16.776157+0000","last_clean":"2024-09-11T22:13:16.776157+0000","last_became_active":"2024-09-11T22:12:10.199280+0000","last_became_peered":"2024-09-11T22:12:10.199280+0000","last_unstale":"2024-09-11T22:13:16.776157+0000","last_undegraded":"2024-09-11T22:13:16.776157+0000","last_fullsized":"2024-09-11T22:13:16.776157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:08.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781626+0000","last_change":"2024-09-11T22:12:02.769477+0000","last_active":"2024-09-11T22:13:16.781626+0000","last_peered":"2024-09-11T22:13:16.781626+0000","last_clean":"2024-09-11T22:13:16.781626+0000","last_became_active":"2024-09-11T22:12:02.768847+0000","last_became_peered":"2024-09-11T22:12:02.768847+0000","last_unstale":"2024-09-11T22:13:16.781626+0000","last_undegraded":"2024-09-11T22:13:16.781626+0000","last_fullsized":"2024-09-11T22:13:16.781626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:57.612115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":91,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:27.218899+0000","last_change":"2024-09-11T22:12:00.744559+0000","last_active":"2024-09-11T22:13:27.218899+0000","last_peered":"2024-09-11T22:13:27.218899+0000","last_clean":"2024-09-11T22:13:27.218899+0000","last_became_active":"2024-09-11T22:12:00.744266+0000","last_became_peered":"2024-09-11T22:12:00.744266+0000","last_unstale":"2024-09-11T22:13:27.218899+0000","last_undegraded":"2024-09-11T22:13:27.218899+0000","last_fullsized":"2024-09-11T22:13:27.218899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:12.663275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037732+0000","last_change":"2024-09-11T22:12:09.819627+0000","last_active":"2024-09-11T22:13:17.037732+0000","last_peered":"2024-09-11T22:13:17.037732+0000","last_clean":"2024-09-11T22:13:17.037732+0000","last_became_active":"2024-09-11T22:12:09.819241+0000","last_became_peered":"2024-09-11T22:12:09.819241+0000","last_unstale":"2024-09-11T22:13:17.037732+0000","last_undegraded":"2024-09-11T22:13:17.037732+0000","last_fullsized":"2024-09-11T22:13:17.037732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:32.932384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782717+0000","last_change":"2024-09-11T22:12:02.776860+0000","last_active":"2024-09-11T22:13:16.782717+0000","last_peered":"2024-09-11T22:13:16.782717+0000","last_clean":"2024-09-11T22:13:16.782717+0000","last_became_active":"2024-09-11T22:12:02.776421+0000","last_became_peered":"2024-09-11T22:12:02.776421+0000","last_unstale":"2024-09-11T22:13:16.782717+0000","last_undegraded":"2024-09-11T22:13:16.782717+0000","last_fullsized":"2024-09-11T22:13:16.782717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:23:43.022309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":89,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:27.218462+0000","last_change":"2024-09-11T22:12:00.817499+0000","last_active":"2024-09-11T22:13:27.218462+0000","last_peered":"2024-09-11T22:13:27.218462+0000","last_clean":"2024-09-11T22:13:27.218462+0000","last_became_active":"2024-09-11T22:12:00.817152+0000","last_became_peered":"2024-09-11T22:12:00.817152+0000","last_unstale":"2024-09-11T22:13:27.218462+0000","last_undegraded":"2024-09-11T22:13:27.218462+0000","last_fullsized":"2024-09-11T22:13:27.218462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:19.716267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038078+0000","last_change":"2024-09-11T22:12:10.200318+0000","last_active":"2024-09-11T22:13:17.038078+0000","last_peered":"2024-09-11T22:13:17.038078+0000","last_clean":"2024-09-11T22:13:17.038078+0000","last_became_active":"2024-09-11T22:12:10.199904+0000","last_became_peered":"2024-09-11T22:12:10.199904+0000","last_unstale":"2024-09-11T22:13:17.038078+0000","last_undegraded":"2024-09-11T22:13:17.038078+0000","last_fullsized":"2024-09-11T22:13:17.038078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:47.061489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037725+0000","last_change":"2024-09-11T22:12:02.758872+0000","last_active":"2024-09-11T22:13:17.037725+0000","last_peered":"2024-09-11T22:13:17.037725+0000","last_clean":"2024-09-11T22:13:17.037725+0000","last_became_active":"2024-09-11T22:12:02.758578+0000","last_became_peered":"2024-09-11T22:12:02.758578+0000","last_unstale":"2024-09-11T22:13:17.037725+0000","last_undegraded":"2024-09-11T22:13:17.037725+0000","last_fullsized":"2024-09-11T22:13:17.037725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:51.875760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779117+0000","last_change":"2024-09-11T22:12:00.749202+0000","last_active":"2024-09-11T22:13:16.779117+0000","last_peered":"2024-09-11T22:13:16.779117+0000","last_clean":"2024-09-11T22:13:16.779117+0000","last_became_active":"2024-09-11T22:12:00.748886+0000","last_became_peered":"2024-09-11T22:12:00.748886+0000","last_unstale":"2024-09-11T22:13:16.779117+0000","last_undegraded":"2024-09-11T22:13:16.779117+0000","last_fullsized":"2024-09-11T22:13:16.779117+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:44.581466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784224+0000","last_change":"2024-09-11T22:12:10.201018+0000","last_active":"2024-09-11T22:13:16.784224+0000","last_peered":"2024-09-11T22:13:16.784224+0000","last_clean":"2024-09-11T22:13:16.784224+0000","last_became_active":"2024-09-11T22:12:10.200658+0000","last_became_peered":"2024-09-11T22:12:10.200658+0000","last_unstale":"2024-09-11T22:13:16.784224+0000","last_undegraded":"2024-09-11T22:13:16.784224+0000","last_fullsized":"2024-09-11T22:13:16.784224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.919652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037964+0000","last_change":"2024-09-11T22:12:02.771546+0000","last_active":"2024-09-11T22:13:17.037964+0000","last_peered":"2024-09-11T22:13:17.037964+0000","last_clean":"2024-09-11T22:13:17.037964+0000","last_became_active":"2024-09-11T22:12:02.771296+0000","last_became_peered":"2024-09-11T22:12:02.771296+0000","last_unstale":"2024-09-11T22:13:17.037964+0000","last_undegraded":"2024-09-11T22:13:17.037964+0000","last_fullsized":"2024-09-11T22:13:17.037964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:05.144166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776881+0000","last_change":"2024-09-11T22:12:00.744431+0000","last_active":"2024-09-11T22:13:16.776881+0000","last_peered":"2024-09-11T22:13:16.776881+0000","last_clean":"2024-09-11T22:13:16.776881+0000","last_became_active":"2024-09-11T22:12:00.744121+0000","last_became_peered":"2024-09-11T22:12:00.744121+0000","last_unstale":"2024-09-11T22:13:16.776881+0000","last_undegraded":"2024-09-11T22:13:16.776881+0000","last_fullsized":"2024-09-11T22:13:16.776881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:16.832914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779230+0000","last_change":"2024-09-11T22:10:51.241028+0000","last_active":"2024-09-11T22:13:16.779230+0000","last_peered":"2024-09-11T22:13:16.779230+0000","last_clean":"2024-09-11T22:13:16.779230+0000","last_became_active":"2024-09-11T22:10:51.240533+0000","last_became_peered":"2024-09-11T22:10:51.240533+0000","last_unstale":"2024-09-11T22:13:16.779230+0000","last_undegraded":"2024-09-11T22:13:16.779230+0000","last_fullsized":"2024-09-11T22:13:16.779230+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_clean_scrub_stamp":"2024-09-11T22:07:30.402427+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:59.522909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779952+0000","last_change":"2024-09-11T22:12:09.815386+0000","last_active":"2024-09-11T22:13:16.779952+0000","last_peered":"2024-09-11T22:13:16.779952+0000","last_clean":"2024-09-11T22:13:16.779952+0000","last_became_active":"2024-09-11T22:12:09.815004+0000","last_became_peered":"2024-09-11T22:12:09.815004+0000","last_unstale":"2024-09-11T22:13:16.779952+0000","last_undegraded":"2024-09-11T22:13:16.779952+0000","last_fullsized":"2024-09-11T22:13:16.779952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:22.247309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780179+0000","last_change":"2024-09-11T22:12:02.774139+0000","last_active":"2024-09-11T22:13:16.780179+0000","last_peered":"2024-09-11T22:13:16.780179+0000","last_clean":"2024-09-11T22:13:16.780179+0000","last_became_active":"2024-09-11T22:12:02.773785+0000","last_became_peered":"2024-09-11T22:12:02.773785+0000","last_unstale":"2024-09-11T22:13:16.780179+0000","last_undegraded":"2024-09-11T22:13:16.780179+0000","last_fullsized":"2024-09-11T22:13:16.780179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:53.160278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:27.238170+0000","last_change":"2024-09-11T22:12:00.818813+0000","last_active":"2024-09-11T22:13:27.238170+0000","last_peered":"2024-09-11T22:13:27.238170+0000","last_clean":"2024-09-11T22:13:27.238170+0000","last_became_active":"2024-09-11T22:12:00.817574+0000","last_became_peered":"2024-09-11T22:12:00.817574+0000","last_unstale":"2024-09-11T22:13:27.238170+0000","last_undegraded":"2024-09-11T22:13:27.238170+0000","last_fullsized":"2024-09-11T22:13:27.238170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:52.429428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784870+0000","last_change":"2024-09-11T22:12:09.819009+0000","last_active":"2024-09-11T22:13:16.784870+0000","last_peered":"2024-09-11T22:13:16.784870+0000","last_clean":"2024-09-11T22:13:16.784870+0000","last_became_active":"2024-09-11T22:12:09.818513+0000","last_became_peered":"2024-09-11T22:12:09.818513+0000","last_unstale":"2024-09-11T22:13:16.784870+0000","last_undegraded":"2024-09-11T22:13:16.784870+0000","last_fullsized":"2024-09-11T22:13:16.784870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:34.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779778+0000","last_change":"2024-09-11T22:12:02.776027+0000","last_active":"2024-09-11T22:13:16.779778+0000","last_peered":"2024-09-11T22:13:16.779778+0000","last_clean":"2024-09-11T22:13:16.779778+0000","last_became_active":"2024-09-11T22:12:02.775485+0000","last_became_peered":"2024-09-11T22:12:02.775485+0000","last_unstale":"2024-09-11T22:13:16.779778+0000","last_undegraded":"2024-09-11T22:13:16.779778+0000","last_fullsized":"2024-09-11T22:13:16.779778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:02.801205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784564+0000","last_change":"2024-09-11T22:12:00.747291+0000","last_active":"2024-09-11T22:13:16.784564+0000","last_peered":"2024-09-11T22:13:16.784564+0000","last_clean":"2024-09-11T22:13:16.784564+0000","last_became_active":"2024-09-11T22:12:00.746910+0000","last_became_peered":"2024-09-11T22:12:00.746910+0000","last_unstale":"2024-09-11T22:13:16.784564+0000","last_undegraded":"2024-09-11T22:13:16.784564+0000","last_fullsized":"2024-09-11T22:13:16.784564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:23.993924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786426+0000","last_change":"2024-09-11T22:13:16.786426+0000","last_active":"2024-09-11T22:13:16.786426+0000","last_peered":"2024-09-11T22:13:16.786426+0000","last_clean":"2024-09-11T22:13:16.786426+0000","last_became_active":"2024-09-11T22:13:16.785964+0000","last_became_peered":"2024-09-11T22:13:16.785964+0000","last_unstale":"2024-09-11T22:13:16.786426+0000","last_undegraded":"2024-09-11T22:13:16.786426+0000","last_fullsized":"2024-09-11T22:13:16.786426+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:48.265620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779815+0000","last_change":"2024-09-11T22:12:02.766451+0000","last_active":"2024-09-11T22:13:16.779815+0000","last_peered":"2024-09-11T22:13:16.779815+0000","last_clean":"2024-09-11T22:13:16.779815+0000","last_became_active":"2024-09-11T22:12:02.765909+0000","last_became_peered":"2024-09-11T22:12:02.765909+0000","last_unstale":"2024-09-11T22:13:16.779815+0000","last_undegraded":"2024-09-11T22:13:16.779815+0000","last_fullsized":"2024-09-11T22:13:16.779815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:34.099196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038072+0000","last_change":"2024-09-11T22:12:00.748989+0000","last_active":"2024-09-11T22:13:17.038072+0000","last_peered":"2024-09-11T22:13:17.038072+0000","last_clean":"2024-09-11T22:13:17.038072+0000","last_became_active":"2024-09-11T22:12:00.748653+0000","last_became_peered":"2024-09-11T22:12:00.748653+0000","last_unstale":"2024-09-11T22:13:17.038072+0000","last_undegraded":"2024-09-11T22:13:17.038072+0000","last_fullsized":"2024-09-11T22:13:17.038072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:53.234719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782374+0000","last_change":"2024-09-11T22:12:09.818187+0000","last_active":"2024-09-11T22:13:16.782374+0000","last_peered":"2024-09-11T22:13:16.782374+0000","last_clean":"2024-09-11T22:13:16.782374+0000","last_became_active":"2024-09-11T22:12:09.817355+0000","last_became_peered":"2024-09-11T22:12:09.817355+0000","last_unstale":"2024-09-11T22:13:16.782374+0000","last_undegraded":"2024-09-11T22:13:16.782374+0000","last_fullsized":"2024-09-11T22:13:16.782374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:57.355716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779302+0000","last_change":"2024-09-11T22:12:02.774309+0000","last_active":"2024-09-11T22:13:16.779302+0000","last_peered":"2024-09-11T22:13:16.779302+0000","last_clean":"2024-09-11T22:13:16.779302+0000","last_became_active":"2024-09-11T22:12:02.774004+0000","last_became_peered":"2024-09-11T22:12:02.774004+0000","last_unstale":"2024-09-11T22:13:16.779302+0000","last_undegraded":"2024-09-11T22:13:16.779302+0000","last_fullsized":"2024-09-11T22:13:16.779302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:03.107880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.791515+0000","last_change":"2024-09-11T22:13:16.791017+0000","last_active":"2024-09-11T22:13:16.791515+0000","last_peered":"2024-09-11T22:13:16.791515+0000","last_clean":"2024-09-11T22:13:16.791515+0000","last_became_active":"2024-09-11T22:13:16.784755+0000","last_became_peered":"2024-09-11T22:13:16.784755+0000","last_unstale":"2024-09-11T22:13:16.791515+0000","last_undegraded":"2024-09-11T22:13:16.791515+0000","last_fullsized":"2024-09-11T22:13:16.791515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:53.317504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.783171+0000","last_change":"2024-09-11T22:12:09.821945+0000","last_active":"2024-09-11T22:13:16.783171+0000","last_peered":"2024-09-11T22:13:16.783171+0000","last_clean":"2024-09-11T22:13:16.783171+0000","last_became_active":"2024-09-11T22:12:09.821537+0000","last_became_peered":"2024-09-11T22:12:09.821537+0000","last_unstale":"2024-09-11T22:13:16.783171+0000","last_undegraded":"2024-09-11T22:13:16.783171+0000","last_fullsized":"2024-09-11T22:13:16.783171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:18.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.783348+0000","last_change":"2024-09-11T22:12:02.768511+0000","last_active":"2024-09-11T22:13:16.783348+0000","last_peered":"2024-09-11T22:13:16.783348+0000","last_clean":"2024-09-11T22:13:16.783348+0000","last_became_active":"2024-09-11T22:12:02.768060+0000","last_became_peered":"2024-09-11T22:12:02.768060+0000","last_unstale":"2024-09-11T22:13:16.783348+0000","last_undegraded":"2024-09-11T22:13:16.783348+0000","last_fullsized":"2024-09-11T22:13:16.783348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:26.417066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776143+0000","last_change":"2024-09-11T22:12:00.817941+0000","last_active":"2024-09-11T22:13:16.776143+0000","last_peered":"2024-09-11T22:13:16.776143+0000","last_clean":"2024-09-11T22:13:16.776143+0000","last_became_active":"2024-09-11T22:12:00.817606+0000","last_became_peered":"2024-09-11T22:12:00.817606+0000","last_unstale":"2024-09-11T22:13:16.776143+0000","last_undegraded":"2024-09-11T22:13:16.776143+0000","last_fullsized":"2024-09-11T22:13:16.776143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:44.212961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038195+0000","last_change":"2024-09-11T22:12:09.812516+0000","last_active":"2024-09-11T22:13:17.038195+0000","last_peered":"2024-09-11T22:13:17.038195+0000","last_clean":"2024-09-11T22:13:17.038195+0000","last_became_active":"2024-09-11T22:12:09.812217+0000","last_became_peered":"2024-09-11T22:12:09.812217+0000","last_unstale":"2024-09-11T22:13:17.038195+0000","last_undegraded":"2024-09-11T22:13:17.038195+0000","last_fullsized":"2024-09-11T22:13:17.038195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:36.568803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778984+0000","last_change":"2024-09-11T22:12:02.775797+0000","last_active":"2024-09-11T22:13:16.778984+0000","last_peered":"2024-09-11T22:13:16.778984+0000","last_clean":"2024-09-11T22:13:16.778984+0000","last_became_active":"2024-09-11T22:12:02.775474+0000","last_became_peered":"2024-09-11T22:12:02.775474+0000","last_unstale":"2024-09-11T22:13:16.778984+0000","last_undegraded":"2024-09-11T22:13:16.778984+0000","last_fullsized":"2024-09-11T22:13:16.778984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:25.446633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":91,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:26.217815+0000","last_change":"2024-09-11T22:12:00.744237+0000","last_active":"2024-09-11T22:13:26.217815+0000","last_peered":"2024-09-11T22:13:26.217815+0000","last_clean":"2024-09-11T22:13:26.217815+0000","last_became_active":"2024-09-11T22:12:00.743783+0000","last_became_peered":"2024-09-11T22:12:00.743783+0000","last_unstale":"2024-09-11T22:13:26.217815+0000","last_undegraded":"2024-09-11T22:13:26.217815+0000","last_fullsized":"2024-09-11T22:13:26.217815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:53.046685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775794+0000","last_change":"2024-09-11T22:12:09.815279+0000","last_active":"2024-09-11T22:13:16.775794+0000","last_peered":"2024-09-11T22:13:16.775794+0000","last_clean":"2024-09-11T22:13:16.775794+0000","last_became_active":"2024-09-11T22:12:09.814447+0000","last_became_peered":"2024-09-11T22:12:09.814447+0000","last_unstale":"2024-09-11T22:13:16.775794+0000","last_undegraded":"2024-09-11T22:13:16.775794+0000","last_fullsized":"2024-09-11T22:13:16.775794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.289292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.039259+0000","last_change":"2024-09-11T22:13:17.039259+0000","last_active":"2024-09-11T22:13:17.039259+0000","last_peered":"2024-09-11T22:13:17.039259+0000","last_clean":"2024-09-11T22:13:17.039259+0000","last_became_active":"2024-09-11T22:13:17.038840+0000","last_became_peered":"2024-09-11T22:13:17.038840+0000","last_unstale":"2024-09-11T22:13:17.039259+0000","last_undegraded":"2024-09-11T22:13:17.039259+0000","last_fullsized":"2024-09-11T22:13:17.039259+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:55.538975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.783817+0000","last_change":"2024-09-11T22:12:00.749008+0000","last_active":"2024-09-11T22:13:16.783817+0000","last_peered":"2024-09-11T22:13:16.783817+0000","last_clean":"2024-09-11T22:13:16.783817+0000","last_became_active":"2024-09-11T22:12:00.748642+0000","last_became_peered":"2024-09-11T22:12:00.748642+0000","last_unstale":"2024-09-11T22:13:16.783817+0000","last_undegraded":"2024-09-11T22:13:16.783817+0000","last_fullsized":"2024-09-11T22:13:16.783817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:34.763101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778735+0000","last_change":"2024-09-11T22:12:09.816925+0000","last_active":"2024-09-11T22:13:16.778735+0000","last_peered":"2024-09-11T22:13:16.778735+0000","last_clean":"2024-09-11T22:13:16.778735+0000","last_became_active":"2024-09-11T22:12:09.816434+0000","last_became_peered":"2024-09-11T22:12:09.816434+0000","last_unstale":"2024-09-11T22:13:16.778735+0000","last_undegraded":"2024-09-11T22:13:16.778735+0000","last_fullsized":"2024-09-11T22:13:16.778735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037551+0000","last_change":"2024-09-11T22:12:02.761721+0000","last_active":"2024-09-11T22:13:17.037551+0000","last_peered":"2024-09-11T22:13:17.037551+0000","last_clean":"2024-09-11T22:13:17.037551+0000","last_became_active":"2024-09-11T22:12:02.761448+0000","last_became_peered":"2024-09-11T22:12:02.761448+0000","last_unstale":"2024-09-11T22:13:17.037551+0000","last_undegraded":"2024-09-11T22:13:17.037551+0000","last_fullsized":"2024-09-11T22:13:17.037551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:51.401729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781599+0000","last_change":"2024-09-11T22:12:00.751110+0000","last_active":"2024-09-11T22:13:16.781599+0000","last_peered":"2024-09-11T22:13:16.781599+0000","last_clean":"2024-09-11T22:13:16.781599+0000","last_became_active":"2024-09-11T22:12:00.749431+0000","last_became_peered":"2024-09-11T22:12:00.749431+0000","last_unstale":"2024-09-11T22:13:16.781599+0000","last_undegraded":"2024-09-11T22:13:16.781599+0000","last_fullsized":"2024-09-11T22:13:16.781599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:27.774865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784188+0000","last_change":"2024-09-11T22:12:09.820852+0000","last_active":"2024-09-11T22:13:16.784188+0000","last_peered":"2024-09-11T22:13:16.784188+0000","last_clean":"2024-09-11T22:13:16.784188+0000","last_became_active":"2024-09-11T22:12:09.820263+0000","last_became_peered":"2024-09-11T22:12:09.820263+0000","last_unstale":"2024-09-11T22:13:16.784188+0000","last_undegraded":"2024-09-11T22:13:16.784188+0000","last_fullsized":"2024-09-11T22:13:16.784188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:09.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776776+0000","last_change":"2024-09-11T22:12:02.770819+0000","last_active":"2024-09-11T22:13:16.776776+0000","last_peered":"2024-09-11T22:13:16.776776+0000","last_clean":"2024-09-11T22:13:16.776776+0000","last_became_active":"2024-09-11T22:12:02.770502+0000","last_became_peered":"2024-09-11T22:12:02.770502+0000","last_unstale":"2024-09-11T22:13:16.776776+0000","last_undegraded":"2024-09-11T22:13:16.776776+0000","last_fullsized":"2024-09-11T22:13:16.776776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:10.345880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":196,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:28.805541+0000","last_change":"2024-09-11T22:13:16.798183+0000","last_active":"2024-09-11T22:13:28.805541+0000","last_peered":"2024-09-11T22:13:28.805541+0000","last_clean":"2024-09-11T22:13:28.805541+0000","last_became_active":"2024-09-11T22:13:16.787422+0000","last_became_peered":"2024-09-11T22:13:16.787422+0000","last_unstale":"2024-09-11T22:13:28.805541+0000","last_undegraded":"2024-09-11T22:13:28.805541+0000","last_fullsized":"2024-09-11T22:13:28.805541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:15.849975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63484,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":112,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":63484,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775334+0000","last_change":"2024-09-11T22:12:09.819643+0000","last_active":"2024-09-11T22:13:16.775334+0000","last_peered":"2024-09-11T22:13:16.775334+0000","last_clean":"2024-09-11T22:13:16.775334+0000","last_became_active":"2024-09-11T22:12:09.819144+0000","last_became_peered":"2024-09-11T22:12:09.819144+0000","last_unstale":"2024-09-11T22:13:16.775334+0000","last_undegraded":"2024-09-11T22:13:16.775334+0000","last_fullsized":"2024-09-11T22:13:16.775334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.006995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784964+0000","last_change":"2024-09-11T22:12:02.762553+0000","last_active":"2024-09-11T22:13:16.784964+0000","last_peered":"2024-09-11T22:13:16.784964+0000","last_clean":"2024-09-11T22:13:16.784964+0000","last_became_active":"2024-09-11T22:12:02.762197+0000","last_became_peered":"2024-09-11T22:12:02.762197+0000","last_unstale":"2024-09-11T22:13:16.784964+0000","last_undegraded":"2024-09-11T22:13:16.784964+0000","last_fullsized":"2024-09-11T22:13:16.784964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:07.188949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784508+0000","last_change":"2024-09-11T22:12:00.817959+0000","last_active":"2024-09-11T22:13:16.784508+0000","last_peered":"2024-09-11T22:13:16.784508+0000","last_clean":"2024-09-11T22:13:16.784508+0000","last_became_active":"2024-09-11T22:12:00.817615+0000","last_became_peered":"2024-09-11T22:12:00.817615+0000","last_unstale":"2024-09-11T22:13:16.784508+0000","last_undegraded":"2024-09-11T22:13:16.784508+0000","last_fullsized":"2024-09-11T22:13:16.784508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:33.671580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786618+0000","last_change":"2024-09-11T22:13:16.786618+0000","last_active":"2024-09-11T22:13:16.786618+0000","last_peered":"2024-09-11T22:13:16.786618+0000","last_clean":"2024-09-11T22:13:16.786618+0000","last_became_active":"2024-09-11T22:13:16.786052+0000","last_became_peered":"2024-09-11T22:13:16.786052+0000","last_unstale":"2024-09-11T22:13:16.786618+0000","last_undegraded":"2024-09-11T22:13:16.786618+0000","last_fullsized":"2024-09-11T22:13:16.786618+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:19.092080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778314+0000","last_change":"2024-09-11T22:12:02.773369+0000","last_active":"2024-09-11T22:13:16.778314+0000","last_peered":"2024-09-11T22:13:16.778314+0000","last_clean":"2024-09-11T22:13:16.778314+0000","last_became_active":"2024-09-11T22:12:02.773046+0000","last_became_peered":"2024-09-11T22:12:02.773046+0000","last_unstale":"2024-09-11T22:13:16.778314+0000","last_undegraded":"2024-09-11T22:13:16.778314+0000","last_fullsized":"2024-09-11T22:13:16.778314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:04.993273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780135+0000","last_change":"2024-09-11T22:12:00.751696+0000","last_active":"2024-09-11T22:13:16.780135+0000","last_peered":"2024-09-11T22:13:16.780135+0000","last_clean":"2024-09-11T22:13:16.780135+0000","last_became_active":"2024-09-11T22:12:00.751362+0000","last_became_peered":"2024-09-11T22:12:00.751362+0000","last_unstale":"2024-09-11T22:13:16.780135+0000","last_undegraded":"2024-09-11T22:13:16.780135+0000","last_fullsized":"2024-09-11T22:13:16.780135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:19.372942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775149+0000","last_change":"2024-09-11T22:12:09.814219+0000","last_active":"2024-09-11T22:13:16.775149+0000","last_peered":"2024-09-11T22:13:16.775149+0000","last_clean":"2024-09-11T22:13:16.775149+0000","last_became_active":"2024-09-11T22:12:09.813817+0000","last_became_peered":"2024-09-11T22:12:09.813817+0000","last_unstale":"2024-09-11T22:13:16.775149+0000","last_undegraded":"2024-09-11T22:13:16.775149+0000","last_fullsized":"2024-09-11T22:13:16.775149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:36.473171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.785654+0000","last_change":"2024-09-11T22:12:02.769739+0000","last_active":"2024-09-11T22:13:16.785654+0000","last_peered":"2024-09-11T22:13:16.785654+0000","last_clean":"2024-09-11T22:13:16.785654+0000","last_became_active":"2024-09-11T22:12:02.769118+0000","last_became_peered":"2024-09-11T22:12:02.769118+0000","last_unstale":"2024-09-11T22:13:16.785654+0000","last_undegraded":"2024-09-11T22:13:16.785654+0000","last_fullsized":"2024-09-11T22:13:16.785654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:01.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:29.224282+0000","last_change":"2024-09-11T22:12:00.748294+0000","last_active":"2024-09-11T22:13:29.224282+0000","last_peered":"2024-09-11T22:13:29.224282+0000","last_clean":"2024-09-11T22:13:29.224282+0000","last_became_active":"2024-09-11T22:12:00.748000+0000","last_became_peered":"2024-09-11T22:12:00.748000+0000","last_unstale":"2024-09-11T22:13:29.224282+0000","last_undegraded":"2024-09-11T22:13:29.224282+0000","last_fullsized":"2024-09-11T22:13:29.224282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:08.629756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037471+0000","last_change":"2024-09-11T22:12:09.820051+0000","last_active":"2024-09-11T22:13:17.037471+0000","last_peered":"2024-09-11T22:13:17.037471+0000","last_clean":"2024-09-11T22:13:17.037471+0000","last_became_active":"2024-09-11T22:12:09.819655+0000","last_became_peered":"2024-09-11T22:12:09.819655+0000","last_unstale":"2024-09-11T22:13:17.037471+0000","last_undegraded":"2024-09-11T22:13:17.037471+0000","last_fullsized":"2024-09-11T22:13:17.037471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:26.560012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777784+0000","last_change":"2024-09-11T22:12:09.812991+0000","last_active":"2024-09-11T22:13:16.777784+0000","last_peered":"2024-09-11T22:13:16.777784+0000","last_clean":"2024-09-11T22:13:16.777784+0000","last_became_active":"2024-09-11T22:12:09.812602+0000","last_became_peered":"2024-09-11T22:12:09.812602+0000","last_unstale":"2024-09-11T22:13:16.777784+0000","last_undegraded":"2024-09-11T22:13:16.777784+0000","last_fullsized":"2024-09-11T22:13:16.777784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:21.198192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775258+0000","last_change":"2024-09-11T22:12:02.760545+0000","last_active":"2024-09-11T22:13:16.775258+0000","last_peered":"2024-09-11T22:13:16.775258+0000","last_clean":"2024-09-11T22:13:16.775258+0000","last_became_active":"2024-09-11T22:12:02.760061+0000","last_became_peered":"2024-09-11T22:12:02.760061+0000","last_unstale":"2024-09-11T22:13:16.775258+0000","last_undegraded":"2024-09-11T22:13:16.775258+0000","last_fullsized":"2024-09-11T22:13:16.775258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:37.000552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782907+0000","last_change":"2024-09-11T22:12:00.750643+0000","last_active":"2024-09-11T22:13:16.782907+0000","last_peered":"2024-09-11T22:13:16.782907+0000","last_clean":"2024-09-11T22:13:16.782907+0000","last_became_active":"2024-09-11T22:12:00.750129+0000","last_became_peered":"2024-09-11T22:12:00.750129+0000","last_unstale":"2024-09-11T22:13:16.782907+0000","last_undegraded":"2024-09-11T22:13:16.782907+0000","last_fullsized":"2024-09-11T22:13:16.782907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:25.519952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786013+0000","last_change":"2024-09-11T22:13:16.786013+0000","last_active":"2024-09-11T22:13:16.786013+0000","last_peered":"2024-09-11T22:13:16.786013+0000","last_clean":"2024-09-11T22:13:16.786013+0000","last_became_active":"2024-09-11T22:13:16.785356+0000","last_became_peered":"2024-09-11T22:13:16.785356+0000","last_unstale":"2024-09-11T22:13:16.786013+0000","last_undegraded":"2024-09-11T22:13:16.786013+0000","last_fullsized":"2024-09-11T22:13:16.786013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:39.411078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037491+0000","last_change":"2024-09-11T22:12:02.766428+0000","last_active":"2024-09-11T22:13:17.037491+0000","last_peered":"2024-09-11T22:13:17.037491+0000","last_clean":"2024-09-11T22:13:17.037491+0000","last_became_active":"2024-09-11T22:12:02.766112+0000","last_became_peered":"2024-09-11T22:12:02.766112+0000","last_unstale":"2024-09-11T22:13:17.037491+0000","last_undegraded":"2024-09-11T22:13:17.037491+0000","last_fullsized":"2024-09-11T22:13:17.037491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:24.701996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037817+0000","last_change":"2024-09-11T22:12:00.743654+0000","last_active":"2024-09-11T22:13:17.037817+0000","last_peered":"2024-09-11T22:13:17.037817+0000","last_clean":"2024-09-11T22:13:17.037817+0000","last_became_active":"2024-09-11T22:12:00.743216+0000","last_became_peered":"2024-09-11T22:12:00.743216+0000","last_unstale":"2024-09-11T22:13:17.037817+0000","last_undegraded":"2024-09-11T22:13:17.037817+0000","last_fullsized":"2024-09-11T22:13:17.037817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.843451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781136+0000","last_change":"2024-09-11T22:12:00.747738+0000","last_active":"2024-09-11T22:13:16.781136+0000","last_peered":"2024-09-11T22:13:16.781136+0000","last_clean":"2024-09-11T22:13:16.781136+0000","last_became_active":"2024-09-11T22:12:00.747424+0000","last_became_peered":"2024-09-11T22:12:00.747424+0000","last_unstale":"2024-09-11T22:13:16.781136+0000","last_undegraded":"2024-09-11T22:13:16.781136+0000","last_fullsized":"2024-09-11T22:13:16.781136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:30.097802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784023+0000","last_change":"2024-09-11T22:12:02.770033+0000","last_active":"2024-09-11T22:13:16.784023+0000","last_peered":"2024-09-11T22:13:16.784023+0000","last_clean":"2024-09-11T22:13:16.784023+0000","last_became_active":"2024-09-11T22:12:02.769597+0000","last_became_peered":"2024-09-11T22:12:02.769597+0000","last_unstale":"2024-09-11T22:13:16.784023+0000","last_undegraded":"2024-09-11T22:13:16.784023+0000","last_fullsized":"2024-09-11T22:13:16.784023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:35.447477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780438+0000","last_change":"2024-09-11T22:12:09.816652+0000","last_active":"2024-09-11T22:13:16.780438+0000","last_peered":"2024-09-11T22:13:16.780438+0000","last_clean":"2024-09-11T22:13:16.780438+0000","last_became_active":"2024-09-11T22:12:09.816259+0000","last_became_peered":"2024-09-11T22:12:09.816259+0000","last_unstale":"2024-09-11T22:13:16.780438+0000","last_undegraded":"2024-09-11T22:13:16.780438+0000","last_fullsized":"2024-09-11T22:13:16.780438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:09.818890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:27.222231+0000","last_change":"2024-09-11T22:12:00.752118+0000","last_active":"2024-09-11T22:13:27.222231+0000","last_peered":"2024-09-11T22:13:27.222231+0000","last_clean":"2024-09-11T22:13:27.222231+0000","last_became_active":"2024-09-11T22:12:00.751757+0000","last_became_peered":"2024-09-11T22:12:00.751757+0000","last_unstale":"2024-09-11T22:13:27.222231+0000","last_undegraded":"2024-09-11T22:13:27.222231+0000","last_fullsized":"2024-09-11T22:13:27.222231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:28.139391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776278+0000","last_change":"2024-09-11T22:12:02.772836+0000","last_active":"2024-09-11T22:13:16.776278+0000","last_peered":"2024-09-11T22:13:16.776278+0000","last_clean":"2024-09-11T22:13:16.776278+0000","last_became_active":"2024-09-11T22:12:02.772528+0000","last_became_peered":"2024-09-11T22:12:02.772528+0000","last_unstale":"2024-09-11T22:13:16.776278+0000","last_undegraded":"2024-09-11T22:13:16.776278+0000","last_fullsized":"2024-09-11T22:13:16.776278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:05.148091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780067+0000","last_change":"2024-09-11T22:12:09.820138+0000","last_active":"2024-09-11T22:13:16.780067+0000","last_peered":"2024-09-11T22:13:16.780067+0000","last_clean":"2024-09-11T22:13:16.780067+0000","last_became_active":"2024-09-11T22:12:09.819428+0000","last_became_peered":"2024-09-11T22:12:09.819428+0000","last_unstale":"2024-09-11T22:13:16.780067+0000","last_undegraded":"2024-09-11T22:13:16.780067+0000","last_fullsized":"2024-09-11T22:13:16.780067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:10.958913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778264+0000","last_change":"2024-09-11T22:12:02.772732+0000","last_active":"2024-09-11T22:13:16.778264+0000","last_peered":"2024-09-11T22:13:16.778264+0000","last_clean":"2024-09-11T22:13:16.778264+0000","last_became_active":"2024-09-11T22:12:02.772407+0000","last_became_peered":"2024-09-11T22:12:02.772407+0000","last_unstale":"2024-09-11T22:13:16.778264+0000","last_undegraded":"2024-09-11T22:13:16.778264+0000","last_fullsized":"2024-09-11T22:13:16.778264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:24.134356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781026+0000","last_change":"2024-09-11T22:12:02.775825+0000","last_active":"2024-09-11T22:13:16.781026+0000","last_peered":"2024-09-11T22:13:16.781026+0000","last_clean":"2024-09-11T22:13:16.781026+0000","last_became_active":"2024-09-11T22:12:02.775471+0000","last_became_peered":"2024-09-11T22:12:02.775471+0000","last_unstale":"2024-09-11T22:13:16.781026+0000","last_undegraded":"2024-09-11T22:13:16.781026+0000","last_fullsized":"2024-09-11T22:13:16.781026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:46.333390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779899+0000","last_change":"2024-09-11T22:12:02.764570+0000","last_active":"2024-09-11T22:13:16.779899+0000","last_peered":"2024-09-11T22:13:16.779899+0000","last_clean":"2024-09-11T22:13:16.779899+0000","last_became_active":"2024-09-11T22:12:02.764005+0000","last_became_peered":"2024-09-11T22:12:02.764005+0000","last_unstale":"2024-09-11T22:13:16.779899+0000","last_undegraded":"2024-09-11T22:13:16.779899+0000","last_fullsized":"2024-09-11T22:13:16.779899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:32.169886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777496+0000","last_change":"2024-09-11T22:12:02.769156+0000","last_active":"2024-09-11T22:13:16.777496+0000","last_peered":"2024-09-11T22:13:16.777496+0000","last_clean":"2024-09-11T22:13:16.777496+0000","last_became_active":"2024-09-11T22:12:02.768795+0000","last_became_peered":"2024-09-11T22:12:02.768795+0000","last_unstale":"2024-09-11T22:13:16.777496+0000","last_undegraded":"2024-09-11T22:13:16.777496+0000","last_fullsized":"2024-09-11T22:13:16.777496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:25.506739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037816+0000","last_change":"2024-09-11T22:12:02.763102+0000","last_active":"2024-09-11T22:13:17.037816+0000","last_peered":"2024-09-11T22:13:17.037816+0000","last_clean":"2024-09-11T22:13:17.037816+0000","last_became_active":"2024-09-11T22:12:02.762687+0000","last_became_peered":"2024-09-11T22:12:02.762687+0000","last_unstale":"2024-09-11T22:13:17.037816+0000","last_undegraded":"2024-09-11T22:13:17.037816+0000","last_fullsized":"2024-09-11T22:13:17.037816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:05.069744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779122+0000","last_change":"2024-09-11T22:12:02.776229+0000","last_active":"2024-09-11T22:13:16.779122+0000","last_peered":"2024-09-11T22:13:16.779122+0000","last_clean":"2024-09-11T22:13:16.779122+0000","last_became_active":"2024-09-11T22:12:02.775875+0000","last_became_peered":"2024-09-11T22:12:02.775875+0000","last_unstale":"2024-09-11T22:13:16.779122+0000","last_undegraded":"2024-09-11T22:13:16.779122+0000","last_fullsized":"2024-09-11T22:13:16.779122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:46.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781971+0000","last_change":"2024-09-11T22:12:02.766783+0000","last_active":"2024-09-11T22:13:16.781971+0000","last_peered":"2024-09-11T22:13:16.781971+0000","last_clean":"2024-09-11T22:13:16.781971+0000","last_became_active":"2024-09-11T22:12:02.766280+0000","last_became_peered":"2024-09-11T22:12:02.766280+0000","last_unstale":"2024-09-11T22:13:16.781971+0000","last_undegraded":"2024-09-11T22:13:16.781971+0000","last_fullsized":"2024-09-11T22:13:16.781971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:12.657682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038187+0000","last_change":"2024-09-11T22:12:02.759008+0000","last_active":"2024-09-11T22:13:17.038187+0000","last_peered":"2024-09-11T22:13:17.038187+0000","last_clean":"2024-09-11T22:13:17.038187+0000","last_became_active":"2024-09-11T22:12:02.758734+0000","last_became_peered":"2024-09-11T22:12:02.758734+0000","last_unstale":"2024-09-11T22:13:17.038187+0000","last_undegraded":"2024-09-11T22:13:17.038187+0000","last_fullsized":"2024-09-11T22:13:17.038187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:37.729891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782612+0000","last_change":"2024-09-11T22:12:02.777260+0000","last_active":"2024-09-11T22:13:16.782612+0000","last_peered":"2024-09-11T22:13:16.782612+0000","last_clean":"2024-09-11T22:13:16.782612+0000","last_became_active":"2024-09-11T22:12:02.775486+0000","last_became_peered":"2024-09-11T22:12:02.775486+0000","last_unstale":"2024-09-11T22:13:16.782612+0000","last_undegraded":"2024-09-11T22:13:16.782612+0000","last_fullsized":"2024-09-11T22:13:16.782612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:41.099229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037635+0000","last_change":"2024-09-11T22:12:02.762025+0000","last_active":"2024-09-11T22:13:17.037635+0000","last_peered":"2024-09-11T22:13:17.037635+0000","last_clean":"2024-09-11T22:13:17.037635+0000","last_became_active":"2024-09-11T22:12:02.761751+0000","last_became_peered":"2024-09-11T22:12:02.761751+0000","last_unstale":"2024-09-11T22:13:17.037635+0000","last_undegraded":"2024-09-11T22:13:17.037635+0000","last_fullsized":"2024-09-11T22:13:17.037635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:45.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.785090+0000","last_change":"2024-09-11T22:12:02.765001+0000","last_active":"2024-09-11T22:13:16.785090+0000","last_peered":"2024-09-11T22:13:16.785090+0000","last_clean":"2024-09-11T22:13:16.785090+0000","last_became_active":"2024-09-11T22:12:02.764667+0000","last_became_peered":"2024-09-11T22:12:02.764667+0000","last_unstale":"2024-09-11T22:13:16.785090+0000","last_undegraded":"2024-09-11T22:13:16.785090+0000","last_fullsized":"2024-09-11T22:13:16.785090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:59.074616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038339+0000","last_change":"2024-09-11T22:12:02.762972+0000","last_active":"2024-09-11T22:13:17.038339+0000","last_peered":"2024-09-11T22:13:17.038339+0000","last_clean":"2024-09-11T22:13:17.038339+0000","last_became_active":"2024-09-11T22:12:02.762685+0000","last_became_peered":"2024-09-11T22:12:02.762685+0000","last_unstale":"2024-09-11T22:13:17.038339+0000","last_undegraded":"2024-09-11T22:13:17.038339+0000","last_fullsized":"2024-09-11T22:13:17.038339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:34.278584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778854+0000","last_change":"2024-09-11T22:12:02.776351+0000","last_active":"2024-09-11T22:13:16.778854+0000","last_peered":"2024-09-11T22:13:16.778854+0000","last_clean":"2024-09-11T22:13:16.778854+0000","last_became_active":"2024-09-11T22:12:02.774827+0000","last_became_peered":"2024-09-11T22:12:02.774827+0000","last_unstale":"2024-09-11T22:13:16.778854+0000","last_undegraded":"2024-09-11T22:13:16.778854+0000","last_fullsized":"2024-09-11T22:13:16.778854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:55.887697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781686+0000","last_change":"2024-09-11T22:12:02.757412+0000","last_active":"2024-09-11T22:13:16.781686+0000","last_peered":"2024-09-11T22:13:16.781686+0000","last_clean":"2024-09-11T22:13:16.781686+0000","last_became_active":"2024-09-11T22:12:02.757013+0000","last_became_peered":"2024-09-11T22:12:02.757013+0000","last_unstale":"2024-09-11T22:13:16.781686+0000","last_undegraded":"2024-09-11T22:13:16.781686+0000","last_fullsized":"2024-09-11T22:13:16.781686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:21.468082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775902+0000","last_change":"2024-09-11T22:12:02.765943+0000","last_active":"2024-09-11T22:13:16.775902+0000","last_peered":"2024-09-11T22:13:16.775902+0000","last_clean":"2024-09-11T22:13:16.775902+0000","last_became_active":"2024-09-11T22:12:02.765617+0000","last_became_peered":"2024-09-11T22:12:02.765617+0000","last_unstale":"2024-09-11T22:13:16.775902+0000","last_undegraded":"2024-09-11T22:13:16.775902+0000","last_fullsized":"2024-09-11T22:13:16.775902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:47.023509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780563+0000","last_change":"2024-09-11T22:12:02.772379+0000","last_active":"2024-09-11T22:13:16.780563+0000","last_peered":"2024-09-11T22:13:16.780563+0000","last_clean":"2024-09-11T22:13:16.780563+0000","last_became_active":"2024-09-11T22:12:02.771561+0000","last_became_peered":"2024-09-11T22:12:02.771561+0000","last_unstale":"2024-09-11T22:13:16.780563+0000","last_undegraded":"2024-09-11T22:13:16.780563+0000","last_fullsized":"2024-09-11T22:13:16.780563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:52.988744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784295+0000","last_change":"2024-09-11T22:12:02.763489+0000","last_active":"2024-09-11T22:13:16.784295+0000","last_peered":"2024-09-11T22:13:16.784295+0000","last_clean":"2024-09-11T22:13:16.784295+0000","last_became_active":"2024-09-11T22:12:02.763194+0000","last_became_peered":"2024-09-11T22:12:02.763194+0000","last_unstale":"2024-09-11T22:13:16.784295+0000","last_undegraded":"2024-09-11T22:13:16.784295+0000","last_fullsized":"2024-09-11T22:13:16.784295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:15:30.685816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777257+0000","last_change":"2024-09-11T22:12:02.768537+0000","last_active":"2024-09-11T22:13:16.777257+0000","last_peered":"2024-09-11T22:13:16.777257+0000","last_clean":"2024-09-11T22:13:16.777257+0000","last_became_active":"2024-09-11T22:12:02.768219+0000","last_became_peered":"2024-09-11T22:12:02.768219+0000","last_unstale":"2024-09-11T22:13:16.777257+0000","last_undegraded":"2024-09-11T22:13:16.777257+0000","last_fullsized":"2024-09-11T22:13:16.777257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:45:03.177587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777507+0000","last_change":"2024-09-11T22:12:02.761823+0000","last_active":"2024-09-11T22:13:16.777507+0000","last_peered":"2024-09-11T22:13:16.777507+0000","last_clean":"2024-09-11T22:13:16.777507+0000","last_became_active":"2024-09-11T22:12:02.761435+0000","last_became_peered":"2024-09-11T22:12:02.761435+0000","last_unstale":"2024-09-11T22:13:16.777507+0000","last_undegraded":"2024-09-11T22:13:16.777507+0000","last_fullsized":"2024-09-11T22:13:16.777507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:54.535178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038341+0000","last_change":"2024-09-11T22:12:02.775613+0000","last_active":"2024-09-11T22:13:17.038341+0000","last_peered":"2024-09-11T22:13:17.038341+0000","last_clean":"2024-09-11T22:13:17.038341+0000","last_became_active":"2024-09-11T22:12:02.775383+0000","last_became_peered":"2024-09-11T22:12:02.775383+0000","last_unstale":"2024-09-11T22:13:17.038341+0000","last_undegraded":"2024-09-11T22:13:17.038341+0000","last_fullsized":"2024-09-11T22:13:17.038341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:18.973047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776869+0000","last_change":"2024-09-11T22:12:02.771155+0000","last_active":"2024-09-11T22:13:16.776869+0000","last_peered":"2024-09-11T22:13:16.776869+0000","last_clean":"2024-09-11T22:13:16.776869+0000","last_became_active":"2024-09-11T22:12:02.770278+0000","last_became_peered":"2024-09-11T22:12:02.770278+0000","last_unstale":"2024-09-11T22:13:16.776869+0000","last_undegraded":"2024-09-11T22:13:16.776869+0000","last_fullsized":"2024-09-11T22:13:16.776869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:23.888372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780165+0000","last_change":"2024-09-11T22:12:02.772840+0000","last_active":"2024-09-11T22:13:16.780165+0000","last_peered":"2024-09-11T22:13:16.780165+0000","last_clean":"2024-09-11T22:13:16.780165+0000","last_became_active":"2024-09-11T22:12:02.772384+0000","last_became_peered":"2024-09-11T22:12:02.772384+0000","last_unstale":"2024-09-11T22:13:16.780165+0000","last_undegraded":"2024-09-11T22:13:16.780165+0000","last_fullsized":"2024-09-11T22:13:16.780165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:42.287294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781051+0000","last_change":"2024-09-11T22:12:02.767741+0000","last_active":"2024-09-11T22:13:16.781051+0000","last_peered":"2024-09-11T22:13:16.781051+0000","last_clean":"2024-09-11T22:13:16.781051+0000","last_became_active":"2024-09-11T22:12:02.767234+0000","last_became_peered":"2024-09-11T22:12:02.767234+0000","last_unstale":"2024-09-11T22:13:16.781051+0000","last_undegraded":"2024-09-11T22:13:16.781051+0000","last_fullsized":"2024-09-11T22:13:16.781051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:22.250763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786025+0000","last_change":"2024-09-11T22:12:02.771265+0000","last_active":"2024-09-11T22:13:16.786025+0000","last_peered":"2024-09-11T22:13:16.786025+0000","last_clean":"2024-09-11T22:13:16.786025+0000","last_became_active":"2024-09-11T22:12:02.770617+0000","last_became_peered":"2024-09-11T22:12:02.770617+0000","last_unstale":"2024-09-11T22:13:16.786025+0000","last_undegraded":"2024-09-11T22:13:16.786025+0000","last_fullsized":"2024-09-11T22:13:16.786025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.390581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037556+0000","last_change":"2024-09-11T22:12:02.770850+0000","last_active":"2024-09-11T22:13:17.037556+0000","last_peered":"2024-09-11T22:13:17.037556+0000","last_clean":"2024-09-11T22:13:17.037556+0000","last_became_active":"2024-09-11T22:12:02.770559+0000","last_became_peered":"2024-09-11T22:12:02.770559+0000","last_unstale":"2024-09-11T22:13:17.037556+0000","last_undegraded":"2024-09-11T22:13:17.037556+0000","last_fullsized":"2024-09-11T22:13:17.037556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:06.865962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780046+0000","last_change":"2024-09-11T22:12:02.775947+0000","last_active":"2024-09-11T22:13:16.780046+0000","last_peered":"2024-09-11T22:13:16.780046+0000","last_clean":"2024-09-11T22:13:16.780046+0000","last_became_active":"2024-09-11T22:12:02.775559+0000","last_became_peered":"2024-09-11T22:12:02.775559+0000","last_unstale":"2024-09-11T22:13:16.780046+0000","last_undegraded":"2024-09-11T22:13:16.780046+0000","last_fullsized":"2024-09-11T22:13:16.780046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:14.642600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781129+0000","last_change":"2024-09-11T22:12:02.768124+0000","last_active":"2024-09-11T22:13:16.781129+0000","last_peered":"2024-09-11T22:13:16.781129+0000","last_clean":"2024-09-11T22:13:16.781129+0000","last_became_active":"2024-09-11T22:12:02.767804+0000","last_became_peered":"2024-09-11T22:12:02.767804+0000","last_unstale":"2024-09-11T22:13:16.781129+0000","last_undegraded":"2024-09-11T22:13:16.781129+0000","last_fullsized":"2024-09-11T22:13:16.781129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:23.612445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776413+0000","last_change":"2024-09-11T22:12:02.764086+0000","last_active":"2024-09-11T22:13:16.776413+0000","last_peered":"2024-09-11T22:13:16.776413+0000","last_clean":"2024-09-11T22:13:16.776413+0000","last_became_active":"2024-09-11T22:12:02.763708+0000","last_became_peered":"2024-09-11T22:12:02.763708+0000","last_unstale":"2024-09-11T22:13:16.776413+0000","last_undegraded":"2024-09-11T22:13:16.776413+0000","last_fullsized":"2024-09-11T22:13:16.776413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:06.058314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782837+0000","last_change":"2024-09-11T22:13:16.782470+0000","last_active":"2024-09-11T22:13:16.782837+0000","last_peered":"2024-09-11T22:13:16.782837+0000","last_clean":"2024-09-11T22:13:16.782837+0000","last_became_active":"2024-09-11T22:13:16.782158+0000","last_became_peered":"2024-09-11T22:13:16.782158+0000","last_unstale":"2024-09-11T22:13:16.782837+0000","last_undegraded":"2024-09-11T22:13:16.782837+0000","last_fullsized":"2024-09-11T22:13:16.782837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:35:37.270023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777779+0000","last_change":"2024-09-11T22:12:02.764702+0000","last_active":"2024-09-11T22:13:16.777779+0000","last_peered":"2024-09-11T22:13:16.777779+0000","last_clean":"2024-09-11T22:13:16.777779+0000","last_became_active":"2024-09-11T22:12:02.764407+0000","last_became_peered":"2024-09-11T22:12:02.764407+0000","last_unstale":"2024-09-11T22:13:16.777779+0000","last_undegraded":"2024-09-11T22:13:16.777779+0000","last_fullsized":"2024-09-11T22:13:16.777779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:01.551823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777891+0000","last_change":"2024-09-11T22:12:02.762209+0000","last_active":"2024-09-11T22:13:16.777891+0000","last_peered":"2024-09-11T22:13:16.777891+0000","last_clean":"2024-09-11T22:13:16.777891+0000","last_became_active":"2024-09-11T22:12:02.761936+0000","last_became_peered":"2024-09-11T22:12:02.761936+0000","last_unstale":"2024-09-11T22:13:16.777891+0000","last_undegraded":"2024-09-11T22:13:16.777891+0000","last_fullsized":"2024-09-11T22:13:16.777891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:39.667680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780446+0000","last_change":"2024-09-11T22:12:02.773537+0000","last_active":"2024-09-11T22:13:16.780446+0000","last_peered":"2024-09-11T22:13:16.780446+0000","last_clean":"2024-09-11T22:13:16.780446+0000","last_became_active":"2024-09-11T22:12:02.773145+0000","last_became_peered":"2024-09-11T22:12:02.773145+0000","last_unstale":"2024-09-11T22:13:16.780446+0000","last_undegraded":"2024-09-11T22:13:16.780446+0000","last_fullsized":"2024-09-11T22:13:16.780446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:30.058536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70692,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":525,"num_read_kb":341,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":63506,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":212076,"data_compressed":14180,"data_compressed_allocated":20480,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286124,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":510321,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678051,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1155072,"data_stored":1160992,"data_compressed":61883,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":514325,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1024000,"data_stored":510585,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1081344,"data_stored":1107499,"data_compressed":60006,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":506126,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691900,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":1102295,"data_compressed":60006,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855426,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":573684,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":126976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117063,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":505441,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1024000,"data_stored":506983,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1073152,"data_stored":570640,"data_compressed":25504,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1032192,"data_stored":507189,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65629,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":64577,"data_compressed":10426,"data_compressed_allocated":12288,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65613,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:13:33.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:13:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:32 smithi110 ceph-mon[34830]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-09-11T22:13:33.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:32 smithi153 ceph-mon[35045]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-09-11T22:13:33.377 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:33.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 -- 172.21.15.92:0/977066092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c101de0 msgr2=0x7fa23c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:33.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 --2- 172.21.15.92:0/977066092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c101de0 0x7fa23c1021c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009900 tx=0x7fa22c02a410 comp rx=0 tx=0).stop 2024-09-11T22:13:33.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 -- 172.21.15.92:0/977066092 shutdown_connections 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 --2- 172.21.15.92:0/977066092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa23c106f10 0x7fa23c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 --2- 172.21.15.92:0/977066092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa23c102790 0x7fa23c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 --2- 172.21.15.92:0/977066092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c101de0 0x7fa23c1021c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 -- 172.21.15.92:0/977066092 >> 172.21.15.92:0/977066092 conn(0x7fa23c0fd8c0 msgr2=0x7fa23c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 -- 172.21.15.92:0/977066092 shutdown_connections 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.861+0000 7fa240abd640 1 -- 172.21.15.92:0/977066092 wait complete. 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 Processor -- start 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 -- start start 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa23c101de0 0x7fa23c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c102790 0x7fa23c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa23c106f10 0x7fa23c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c111de0 con 0x7fa23c102790 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c111c60 con 0x7fa23c106f10 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa240abd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c111f60 con 0x7fa23c101de0 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa233fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c102790 0x7fa23c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa233fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c102790 0x7fa23c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33320/0 (socket says 172.21.15.92:33320) 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.862+0000 7fa233fff640 1 -- 172.21.15.92:0/1493464843 learned_addr learned my addr 172.21.15.92:0/1493464843 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa23a59c640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa23c101de0 0x7fa23c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa23ad9d640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa23c106f10 0x7fa23c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa233fff640 1 -- 172.21.15.92:0/1493464843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa23c101de0 msgr2=0x7fa23c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa233fff640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa23c101de0 0x7fa23c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa233fff640 1 -- 172.21.15.92:0/1493464843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa23c106f10 msgr2=0x7fa23c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa233fff640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa23c106f10 0x7fa23c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa233fff640 1 -- 172.21.15.92:0/1493464843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23c19d330 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa23ad9d640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa23c106f10 0x7fa23c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa233fff640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c102790 0x7fa23c1988c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c890 tx=0x7fa22400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa2337fe640 1 -- 172.21.15.92:0/1493464843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224004220 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa2337fe640 1 -- 172.21.15.92:0/1493464843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2240043c0 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23c19d620 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa2337fe640 1 -- 172.21.15.92:0/1493464843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224004a00 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.863+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23c1a4f00 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.864+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23c1021c0 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.865+0000 7fa2337fe640 1 -- 172.21.15.92:0/1493464843 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa224004ba0 con 0x7fa23c102790 2024-09-11T22:13:33.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.866+0000 7fa2337fe640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2280a2c10 0x7fa2280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:33.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.866+0000 7fa23a59c640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2280a2c10 0x7fa2280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:33.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.866+0000 7fa2337fe640 1 -- 172.21.15.92:0/1493464843 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa2240cf1f0 con 0x7fa23c102790 2024-09-11T22:13:33.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.866+0000 7fa23a59c640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2280a2c10 0x7fa2280a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa22c031440 tx=0x7fa22c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:33.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:33.870+0000 7fa2337fe640 1 -- 172.21.15.92:0/1493464843 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa224014030 con 0x7fa23c102790 2024-09-11T22:13:34.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.035+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa23c062630 con 0x7fa2280a2c10 2024-09-11T22:13:34.036 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:33 smithi092 ceph-mon[31173]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:34.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.043+0000 7fa2337fe640 1 -- 172.21.15.92:0/1493464843 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340726 (secure 0 0 0) 0x7fa23c062630 con 0x7fa2280a2c10 2024-09-11T22:13:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2280a2c10 msgr2=0x7fa2280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2280a2c10 0x7fa2280a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa22c031440 tx=0x7fa22c005b50 comp rx=0 tx=0).stop 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c102790 msgr2=0x7fa23c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c102790 0x7fa23c1988c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c890 tx=0x7fa22400cd50 comp rx=0 tx=0).stop 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 shutdown_connections 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa2280a2c10 0x7fa2280a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa23c106f10 0x7fa23c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa23c102790 0x7fa23c1988c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 --2- 172.21.15.92:0/1493464843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa23c101de0 0x7fa23c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.047+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 >> 172.21.15.92:0/1493464843 conn(0x7fa23c0fd8c0 msgr2=0x7fa23c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.048+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 shutdown_connections 2024-09-11T22:13:34.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:34.048+0000 7fa240abd640 1 -- 172.21.15.92:0/1493464843 wait complete. 2024-09-11T22:13:34.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:33 smithi110 ceph-mon[34830]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:34.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:33 smithi153 ceph-mon[35045]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:34.708 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-11T22:13:33.704495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661294,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":671,"num_read_kb":450,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1244242,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382416,"kb_used_data":12300,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559408,"statfs":{"total":1151940427776,"available":1151548833792,"internally_reserved":0,"allocated":12595200,"data_stored":8076080,"data_compressed":406841,"data_compressed_allocated":548864,"data_compressed_original":2633728,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003212"},"pg_stats":[{"pgid":"2.1f","version":"90'13","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:27.218406+0000","last_change":"2024-09-11T22:12:00.751331+0000","last_active":"2024-09-11T22:13:27.218406+0000","last_peered":"2024-09-11T22:13:27.218406+0000","last_clean":"2024-09-11T22:13:27.218406+0000","last_became_active":"2024-09-11T22:12:00.751034+0000","last_became_peered":"2024-09-11T22:12:00.751034+0000","last_unstale":"2024-09-11T22:13:27.218406+0000","last_undegraded":"2024-09-11T22:13:27.218406+0000","last_fullsized":"2024-09-11T22:13:27.218406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:14.418754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786549+0000","last_change":"2024-09-11T22:13:16.786159+0000","last_active":"2024-09-11T22:13:16.786549+0000","last_peered":"2024-09-11T22:13:16.786549+0000","last_clean":"2024-09-11T22:13:16.786549+0000","last_became_active":"2024-09-11T22:13:16.785880+0000","last_became_peered":"2024-09-11T22:13:16.785880+0000","last_unstale":"2024-09-11T22:13:16.786549+0000","last_undegraded":"2024-09-11T22:13:16.786549+0000","last_fullsized":"2024-09-11T22:13:16.786549+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:54.511343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780480+0000","last_change":"2024-09-11T22:12:09.818054+0000","last_active":"2024-09-11T22:13:16.780480+0000","last_peered":"2024-09-11T22:13:16.780480+0000","last_clean":"2024-09-11T22:13:16.780480+0000","last_became_active":"2024-09-11T22:12:09.817527+0000","last_became_peered":"2024-09-11T22:12:09.817527+0000","last_unstale":"2024-09-11T22:13:16.780480+0000","last_undegraded":"2024-09-11T22:13:16.780480+0000","last_fullsized":"2024-09-11T22:13:16.780480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:26.496214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038623+0000","last_change":"2024-09-11T22:12:00.817451+0000","last_active":"2024-09-11T22:13:17.038623+0000","last_peered":"2024-09-11T22:13:17.038623+0000","last_clean":"2024-09-11T22:13:17.038623+0000","last_became_active":"2024-09-11T22:12:00.817047+0000","last_became_peered":"2024-09-11T22:12:00.817047+0000","last_unstale":"2024-09-11T22:13:17.038623+0000","last_undegraded":"2024-09-11T22:13:17.038623+0000","last_fullsized":"2024-09-11T22:13:17.038623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:48.450520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778306+0000","last_change":"2024-09-11T22:12:02.764638+0000","last_active":"2024-09-11T22:13:16.778306+0000","last_peered":"2024-09-11T22:13:16.778306+0000","last_clean":"2024-09-11T22:13:16.778306+0000","last_became_active":"2024-09-11T22:12:02.764329+0000","last_became_peered":"2024-09-11T22:12:02.764329+0000","last_unstale":"2024-09-11T22:13:16.778306+0000","last_undegraded":"2024-09-11T22:13:16.778306+0000","last_fullsized":"2024-09-11T22:13:16.778306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:27.322976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776076+0000","last_change":"2024-09-11T22:12:09.817478+0000","last_active":"2024-09-11T22:13:16.776076+0000","last_peered":"2024-09-11T22:13:16.776076+0000","last_clean":"2024-09-11T22:13:16.776076+0000","last_became_active":"2024-09-11T22:12:09.817098+0000","last_became_peered":"2024-09-11T22:12:09.817098+0000","last_unstale":"2024-09-11T22:13:16.776076+0000","last_undegraded":"2024-09-11T22:13:16.776076+0000","last_fullsized":"2024-09-11T22:13:16.776076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:50.066572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776891+0000","last_change":"2024-09-11T22:12:00.754280+0000","last_active":"2024-09-11T22:13:16.776891+0000","last_peered":"2024-09-11T22:13:16.776891+0000","last_clean":"2024-09-11T22:13:16.776891+0000","last_became_active":"2024-09-11T22:12:00.753906+0000","last_became_peered":"2024-09-11T22:12:00.753906+0000","last_unstale":"2024-09-11T22:13:16.776891+0000","last_undegraded":"2024-09-11T22:13:16.776891+0000","last_fullsized":"2024-09-11T22:13:16.776891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:01.049033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784841+0000","last_change":"2024-09-11T22:12:02.765037+0000","last_active":"2024-09-11T22:13:16.784841+0000","last_peered":"2024-09-11T22:13:16.784841+0000","last_clean":"2024-09-11T22:13:16.784841+0000","last_became_active":"2024-09-11T22:12:02.764730+0000","last_became_peered":"2024-09-11T22:12:02.764730+0000","last_unstale":"2024-09-11T22:13:16.784841+0000","last_undegraded":"2024-09-11T22:13:16.784841+0000","last_fullsized":"2024-09-11T22:13:16.784841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:13.974389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776396+0000","last_change":"2024-09-11T22:12:09.817801+0000","last_active":"2024-09-11T22:13:16.776396+0000","last_peered":"2024-09-11T22:13:16.776396+0000","last_clean":"2024-09-11T22:13:16.776396+0000","last_became_active":"2024-09-11T22:12:09.817389+0000","last_became_peered":"2024-09-11T22:12:09.817389+0000","last_unstale":"2024-09-11T22:13:16.776396+0000","last_undegraded":"2024-09-11T22:13:16.776396+0000","last_fullsized":"2024-09-11T22:13:16.776396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:55.152113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776816+0000","last_change":"2024-09-11T22:12:00.752216+0000","last_active":"2024-09-11T22:13:16.776816+0000","last_peered":"2024-09-11T22:13:16.776816+0000","last_clean":"2024-09-11T22:13:16.776816+0000","last_became_active":"2024-09-11T22:12:00.751641+0000","last_became_peered":"2024-09-11T22:12:00.751641+0000","last_unstale":"2024-09-11T22:13:16.776816+0000","last_undegraded":"2024-09-11T22:13:16.776816+0000","last_fullsized":"2024-09-11T22:13:16.776816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784299+0000","last_change":"2024-09-11T22:12:02.776766+0000","last_active":"2024-09-11T22:13:16.784299+0000","last_peered":"2024-09-11T22:13:16.784299+0000","last_clean":"2024-09-11T22:13:16.784299+0000","last_became_active":"2024-09-11T22:12:02.775985+0000","last_became_peered":"2024-09-11T22:12:02.775985+0000","last_unstale":"2024-09-11T22:13:16.784299+0000","last_undegraded":"2024-09-11T22:13:16.784299+0000","last_fullsized":"2024-09-11T22:13:16.784299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:39.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776177+0000","last_change":"2024-09-11T22:12:09.817906+0000","last_active":"2024-09-11T22:13:16.776177+0000","last_peered":"2024-09-11T22:13:16.776177+0000","last_clean":"2024-09-11T22:13:16.776177+0000","last_became_active":"2024-09-11T22:12:09.817404+0000","last_became_peered":"2024-09-11T22:12:09.817404+0000","last_unstale":"2024-09-11T22:13:16.776177+0000","last_undegraded":"2024-09-11T22:13:16.776177+0000","last_fullsized":"2024-09-11T22:13:16.776177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:17:29.957890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779692+0000","last_change":"2024-09-11T22:12:02.770932+0000","last_active":"2024-09-11T22:13:16.779692+0000","last_peered":"2024-09-11T22:13:16.779692+0000","last_clean":"2024-09-11T22:13:16.779692+0000","last_became_active":"2024-09-11T22:12:02.770619+0000","last_became_peered":"2024-09-11T22:12:02.770619+0000","last_unstale":"2024-09-11T22:13:16.779692+0000","last_undegraded":"2024-09-11T22:13:16.779692+0000","last_fullsized":"2024-09-11T22:13:16.779692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:31.746892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.791862+0000","last_change":"2024-09-11T22:13:16.791862+0000","last_active":"2024-09-11T22:13:16.791862+0000","last_peered":"2024-09-11T22:13:16.791862+0000","last_clean":"2024-09-11T22:13:16.791862+0000","last_became_active":"2024-09-11T22:13:16.787780+0000","last_became_peered":"2024-09-11T22:13:16.787780+0000","last_unstale":"2024-09-11T22:13:16.791862+0000","last_undegraded":"2024-09-11T22:13:16.791862+0000","last_fullsized":"2024-09-11T22:13:16.791862+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:00.519704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776411+0000","last_change":"2024-09-11T22:12:10.199589+0000","last_active":"2024-09-11T22:13:16.776411+0000","last_peered":"2024-09-11T22:13:16.776411+0000","last_clean":"2024-09-11T22:13:16.776411+0000","last_became_active":"2024-09-11T22:12:10.199185+0000","last_became_peered":"2024-09-11T22:12:10.199185+0000","last_unstale":"2024-09-11T22:13:16.776411+0000","last_undegraded":"2024-09-11T22:13:16.776411+0000","last_fullsized":"2024-09-11T22:13:16.776411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:51.081016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037739+0000","last_change":"2024-09-11T22:12:02.763355+0000","last_active":"2024-09-11T22:13:17.037739+0000","last_peered":"2024-09-11T22:13:17.037739+0000","last_clean":"2024-09-11T22:13:17.037739+0000","last_became_active":"2024-09-11T22:12:02.763060+0000","last_became_peered":"2024-09-11T22:12:02.763060+0000","last_unstale":"2024-09-11T22:13:17.037739+0000","last_undegraded":"2024-09-11T22:13:17.037739+0000","last_fullsized":"2024-09-11T22:13:17.037739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:22.776503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780037+0000","last_change":"2024-09-11T22:12:00.754158+0000","last_active":"2024-09-11T22:13:16.780037+0000","last_peered":"2024-09-11T22:13:16.780037+0000","last_clean":"2024-09-11T22:13:16.780037+0000","last_became_active":"2024-09-11T22:12:00.753706+0000","last_became_peered":"2024-09-11T22:12:00.753706+0000","last_unstale":"2024-09-11T22:13:16.780037+0000","last_undegraded":"2024-09-11T22:13:16.780037+0000","last_fullsized":"2024-09-11T22:13:16.780037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:40.470704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776546+0000","last_change":"2024-09-11T22:12:09.810374+0000","last_active":"2024-09-11T22:13:16.776546+0000","last_peered":"2024-09-11T22:13:16.776546+0000","last_clean":"2024-09-11T22:13:16.776546+0000","last_became_active":"2024-09-11T22:12:09.809915+0000","last_became_peered":"2024-09-11T22:12:09.809915+0000","last_unstale":"2024-09-11T22:13:16.776546+0000","last_undegraded":"2024-09-11T22:13:16.776546+0000","last_fullsized":"2024-09-11T22:13:16.776546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:25.554920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778581+0000","last_change":"2024-09-11T22:12:02.769350+0000","last_active":"2024-09-11T22:13:16.778581+0000","last_peered":"2024-09-11T22:13:16.778581+0000","last_clean":"2024-09-11T22:13:16.778581+0000","last_became_active":"2024-09-11T22:12:02.768896+0000","last_became_peered":"2024-09-11T22:12:02.768896+0000","last_unstale":"2024-09-11T22:13:16.778581+0000","last_undegraded":"2024-09-11T22:13:16.778581+0000","last_fullsized":"2024-09-11T22:13:16.778581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:12.490938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037624+0000","last_change":"2024-09-11T22:12:00.743281+0000","last_active":"2024-09-11T22:13:17.037624+0000","last_peered":"2024-09-11T22:13:17.037624+0000","last_clean":"2024-09-11T22:13:17.037624+0000","last_became_active":"2024-09-11T22:12:00.742977+0000","last_became_peered":"2024-09-11T22:12:00.742977+0000","last_unstale":"2024-09-11T22:13:17.037624+0000","last_undegraded":"2024-09-11T22:13:17.037624+0000","last_fullsized":"2024-09-11T22:13:17.037624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:28.890328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775712+0000","last_change":"2024-09-11T22:12:09.815471+0000","last_active":"2024-09-11T22:13:16.775712+0000","last_peered":"2024-09-11T22:13:16.775712+0000","last_clean":"2024-09-11T22:13:16.775712+0000","last_became_active":"2024-09-11T22:12:09.815098+0000","last_became_peered":"2024-09-11T22:12:09.815098+0000","last_unstale":"2024-09-11T22:13:16.775712+0000","last_undegraded":"2024-09-11T22:13:16.775712+0000","last_fullsized":"2024-09-11T22:13:16.775712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:36.375848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776142+0000","last_change":"2024-09-11T22:12:02.763597+0000","last_active":"2024-09-11T22:13:16.776142+0000","last_peered":"2024-09-11T22:13:16.776142+0000","last_clean":"2024-09-11T22:13:16.776142+0000","last_became_active":"2024-09-11T22:12:02.763242+0000","last_became_peered":"2024-09-11T22:12:02.763242+0000","last_unstale":"2024-09-11T22:13:16.776142+0000","last_undegraded":"2024-09-11T22:13:16.776142+0000","last_fullsized":"2024-09-11T22:13:16.776142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:24.968799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.785828+0000","last_change":"2024-09-11T22:12:00.817604+0000","last_active":"2024-09-11T22:13:16.785828+0000","last_peered":"2024-09-11T22:13:16.785828+0000","last_clean":"2024-09-11T22:13:16.785828+0000","last_became_active":"2024-09-11T22:12:00.817377+0000","last_became_peered":"2024-09-11T22:12:00.817377+0000","last_unstale":"2024-09-11T22:13:16.785828+0000","last_undegraded":"2024-09-11T22:13:16.785828+0000","last_fullsized":"2024-09-11T22:13:16.785828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:27.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781909+0000","last_change":"2024-09-11T22:13:16.781909+0000","last_active":"2024-09-11T22:13:16.781909+0000","last_peered":"2024-09-11T22:13:16.781909+0000","last_clean":"2024-09-11T22:13:16.781909+0000","last_became_active":"2024-09-11T22:13:16.781242+0000","last_became_peered":"2024-09-11T22:13:16.781242+0000","last_unstale":"2024-09-11T22:13:16.781909+0000","last_undegraded":"2024-09-11T22:13:16.781909+0000","last_fullsized":"2024-09-11T22:13:16.781909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:23.724541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777977+0000","last_change":"2024-09-11T22:12:02.773380+0000","last_active":"2024-09-11T22:13:16.777977+0000","last_peered":"2024-09-11T22:13:16.777977+0000","last_clean":"2024-09-11T22:13:16.777977+0000","last_became_active":"2024-09-11T22:12:02.773076+0000","last_became_peered":"2024-09-11T22:12:02.773076+0000","last_unstale":"2024-09-11T22:13:16.777977+0000","last_undegraded":"2024-09-11T22:13:16.777977+0000","last_fullsized":"2024-09-11T22:13:16.777977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:35.475581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777306+0000","last_change":"2024-09-11T22:12:00.746438+0000","last_active":"2024-09-11T22:13:16.777306+0000","last_peered":"2024-09-11T22:13:16.777306+0000","last_clean":"2024-09-11T22:13:16.777306+0000","last_became_active":"2024-09-11T22:12:00.746034+0000","last_became_peered":"2024-09-11T22:12:00.746034+0000","last_unstale":"2024-09-11T22:13:16.777306+0000","last_undegraded":"2024-09-11T22:13:16.777306+0000","last_fullsized":"2024-09-11T22:13:16.777306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:49.156748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780666+0000","last_change":"2024-09-11T22:12:09.812553+0000","last_active":"2024-09-11T22:13:16.780666+0000","last_peered":"2024-09-11T22:13:16.780666+0000","last_clean":"2024-09-11T22:13:16.780666+0000","last_became_active":"2024-09-11T22:12:09.812092+0000","last_became_peered":"2024-09-11T22:12:09.812092+0000","last_unstale":"2024-09-11T22:13:16.780666+0000","last_undegraded":"2024-09-11T22:13:16.780666+0000","last_fullsized":"2024-09-11T22:13:16.780666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:13.947799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038274+0000","last_change":"2024-09-11T22:12:02.763399+0000","last_active":"2024-09-11T22:13:17.038274+0000","last_peered":"2024-09-11T22:13:17.038274+0000","last_clean":"2024-09-11T22:13:17.038274+0000","last_became_active":"2024-09-11T22:12:02.763082+0000","last_became_peered":"2024-09-11T22:12:02.763082+0000","last_unstale":"2024-09-11T22:13:17.038274+0000","last_undegraded":"2024-09-11T22:13:17.038274+0000","last_fullsized":"2024-09-11T22:13:17.038274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:52.267960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775942+0000","last_change":"2024-09-11T22:12:00.751318+0000","last_active":"2024-09-11T22:13:16.775942+0000","last_peered":"2024-09-11T22:13:16.775942+0000","last_clean":"2024-09-11T22:13:16.775942+0000","last_became_active":"2024-09-11T22:12:00.751022+0000","last_became_peered":"2024-09-11T22:12:00.751022+0000","last_unstale":"2024-09-11T22:13:16.775942+0000","last_undegraded":"2024-09-11T22:13:16.775942+0000","last_fullsized":"2024-09-11T22:13:16.775942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:23:26.163502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037272+0000","last_change":"2024-09-11T22:12:10.200279+0000","last_active":"2024-09-11T22:13:17.037272+0000","last_peered":"2024-09-11T22:13:17.037272+0000","last_clean":"2024-09-11T22:13:17.037272+0000","last_became_active":"2024-09-11T22:12:10.199920+0000","last_became_peered":"2024-09-11T22:12:10.199920+0000","last_unstale":"2024-09-11T22:13:17.037272+0000","last_undegraded":"2024-09-11T22:13:17.037272+0000","last_fullsized":"2024-09-11T22:13:17.037272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:59.012735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037898+0000","last_change":"2024-09-11T22:12:02.755565+0000","last_active":"2024-09-11T22:13:17.037898+0000","last_peered":"2024-09-11T22:13:17.037898+0000","last_clean":"2024-09-11T22:13:17.037898+0000","last_became_active":"2024-09-11T22:12:02.755236+0000","last_became_peered":"2024-09-11T22:12:02.755236+0000","last_unstale":"2024-09-11T22:13:17.037898+0000","last_undegraded":"2024-09-11T22:13:17.037898+0000","last_fullsized":"2024-09-11T22:13:17.037898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.588976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776929+0000","last_change":"2024-09-11T22:12:00.818634+0000","last_active":"2024-09-11T22:13:16.776929+0000","last_peered":"2024-09-11T22:13:16.776929+0000","last_clean":"2024-09-11T22:13:16.776929+0000","last_became_active":"2024-09-11T22:12:00.817917+0000","last_became_peered":"2024-09-11T22:12:00.817917+0000","last_unstale":"2024-09-11T22:13:16.776929+0000","last_undegraded":"2024-09-11T22:13:16.776929+0000","last_fullsized":"2024-09-11T22:13:16.776929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:42.382962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778793+0000","last_change":"2024-09-11T22:12:09.814770+0000","last_active":"2024-09-11T22:13:16.778793+0000","last_peered":"2024-09-11T22:13:16.778793+0000","last_clean":"2024-09-11T22:13:16.778793+0000","last_became_active":"2024-09-11T22:12:09.814322+0000","last_became_peered":"2024-09-11T22:12:09.814322+0000","last_unstale":"2024-09-11T22:13:16.778793+0000","last_undegraded":"2024-09-11T22:13:16.778793+0000","last_fullsized":"2024-09-11T22:13:16.778793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.666161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781247+0000","last_change":"2024-09-11T22:12:02.762939+0000","last_active":"2024-09-11T22:13:16.781247+0000","last_peered":"2024-09-11T22:13:16.781247+0000","last_clean":"2024-09-11T22:13:16.781247+0000","last_became_active":"2024-09-11T22:12:02.762579+0000","last_became_peered":"2024-09-11T22:12:02.762579+0000","last_unstale":"2024-09-11T22:13:16.781247+0000","last_undegraded":"2024-09-11T22:13:16.781247+0000","last_fullsized":"2024-09-11T22:13:16.781247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:35.124612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775930+0000","last_change":"2024-09-11T22:12:00.818094+0000","last_active":"2024-09-11T22:13:16.775930+0000","last_peered":"2024-09-11T22:13:16.775930+0000","last_clean":"2024-09-11T22:13:16.775930+0000","last_became_active":"2024-09-11T22:12:00.817723+0000","last_became_peered":"2024-09-11T22:12:00.817723+0000","last_unstale":"2024-09-11T22:13:16.775930+0000","last_undegraded":"2024-09-11T22:13:16.775930+0000","last_fullsized":"2024-09-11T22:13:16.775930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:32.618689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776157+0000","last_change":"2024-09-11T22:12:10.199778+0000","last_active":"2024-09-11T22:13:16.776157+0000","last_peered":"2024-09-11T22:13:16.776157+0000","last_clean":"2024-09-11T22:13:16.776157+0000","last_became_active":"2024-09-11T22:12:10.199280+0000","last_became_peered":"2024-09-11T22:12:10.199280+0000","last_unstale":"2024-09-11T22:13:16.776157+0000","last_undegraded":"2024-09-11T22:13:16.776157+0000","last_fullsized":"2024-09-11T22:13:16.776157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:08.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781626+0000","last_change":"2024-09-11T22:12:02.769477+0000","last_active":"2024-09-11T22:13:16.781626+0000","last_peered":"2024-09-11T22:13:16.781626+0000","last_clean":"2024-09-11T22:13:16.781626+0000","last_became_active":"2024-09-11T22:12:02.768847+0000","last_became_peered":"2024-09-11T22:12:02.768847+0000","last_unstale":"2024-09-11T22:13:16.781626+0000","last_undegraded":"2024-09-11T22:13:16.781626+0000","last_fullsized":"2024-09-11T22:13:16.781626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:57.612115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":96,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:32.224658+0000","last_change":"2024-09-11T22:12:00.744559+0000","last_active":"2024-09-11T22:13:32.224658+0000","last_peered":"2024-09-11T22:13:32.224658+0000","last_clean":"2024-09-11T22:13:32.224658+0000","last_became_active":"2024-09-11T22:12:00.744266+0000","last_became_peered":"2024-09-11T22:12:00.744266+0000","last_unstale":"2024-09-11T22:13:32.224658+0000","last_undegraded":"2024-09-11T22:13:32.224658+0000","last_fullsized":"2024-09-11T22:13:32.224658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:12.663275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037732+0000","last_change":"2024-09-11T22:12:09.819627+0000","last_active":"2024-09-11T22:13:17.037732+0000","last_peered":"2024-09-11T22:13:17.037732+0000","last_clean":"2024-09-11T22:13:17.037732+0000","last_became_active":"2024-09-11T22:12:09.819241+0000","last_became_peered":"2024-09-11T22:12:09.819241+0000","last_unstale":"2024-09-11T22:13:17.037732+0000","last_undegraded":"2024-09-11T22:13:17.037732+0000","last_fullsized":"2024-09-11T22:13:17.037732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:32.932384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782717+0000","last_change":"2024-09-11T22:12:02.776860+0000","last_active":"2024-09-11T22:13:16.782717+0000","last_peered":"2024-09-11T22:13:16.782717+0000","last_clean":"2024-09-11T22:13:16.782717+0000","last_became_active":"2024-09-11T22:12:02.776421+0000","last_became_peered":"2024-09-11T22:12:02.776421+0000","last_unstale":"2024-09-11T22:13:16.782717+0000","last_undegraded":"2024-09-11T22:13:16.782717+0000","last_fullsized":"2024-09-11T22:13:16.782717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:23:43.022309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":89,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:27.218462+0000","last_change":"2024-09-11T22:12:00.817499+0000","last_active":"2024-09-11T22:13:27.218462+0000","last_peered":"2024-09-11T22:13:27.218462+0000","last_clean":"2024-09-11T22:13:27.218462+0000","last_became_active":"2024-09-11T22:12:00.817152+0000","last_became_peered":"2024-09-11T22:12:00.817152+0000","last_unstale":"2024-09-11T22:13:27.218462+0000","last_undegraded":"2024-09-11T22:13:27.218462+0000","last_fullsized":"2024-09-11T22:13:27.218462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:19.716267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038078+0000","last_change":"2024-09-11T22:12:10.200318+0000","last_active":"2024-09-11T22:13:17.038078+0000","last_peered":"2024-09-11T22:13:17.038078+0000","last_clean":"2024-09-11T22:13:17.038078+0000","last_became_active":"2024-09-11T22:12:10.199904+0000","last_became_peered":"2024-09-11T22:12:10.199904+0000","last_unstale":"2024-09-11T22:13:17.038078+0000","last_undegraded":"2024-09-11T22:13:17.038078+0000","last_fullsized":"2024-09-11T22:13:17.038078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:47.061489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037725+0000","last_change":"2024-09-11T22:12:02.758872+0000","last_active":"2024-09-11T22:13:17.037725+0000","last_peered":"2024-09-11T22:13:17.037725+0000","last_clean":"2024-09-11T22:13:17.037725+0000","last_became_active":"2024-09-11T22:12:02.758578+0000","last_became_peered":"2024-09-11T22:12:02.758578+0000","last_unstale":"2024-09-11T22:13:17.037725+0000","last_undegraded":"2024-09-11T22:13:17.037725+0000","last_fullsized":"2024-09-11T22:13:17.037725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:51.875760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779117+0000","last_change":"2024-09-11T22:12:00.749202+0000","last_active":"2024-09-11T22:13:16.779117+0000","last_peered":"2024-09-11T22:13:16.779117+0000","last_clean":"2024-09-11T22:13:16.779117+0000","last_became_active":"2024-09-11T22:12:00.748886+0000","last_became_peered":"2024-09-11T22:12:00.748886+0000","last_unstale":"2024-09-11T22:13:16.779117+0000","last_undegraded":"2024-09-11T22:13:16.779117+0000","last_fullsized":"2024-09-11T22:13:16.779117+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:44.581466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784224+0000","last_change":"2024-09-11T22:12:10.201018+0000","last_active":"2024-09-11T22:13:16.784224+0000","last_peered":"2024-09-11T22:13:16.784224+0000","last_clean":"2024-09-11T22:13:16.784224+0000","last_became_active":"2024-09-11T22:12:10.200658+0000","last_became_peered":"2024-09-11T22:12:10.200658+0000","last_unstale":"2024-09-11T22:13:16.784224+0000","last_undegraded":"2024-09-11T22:13:16.784224+0000","last_fullsized":"2024-09-11T22:13:16.784224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.919652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037964+0000","last_change":"2024-09-11T22:12:02.771546+0000","last_active":"2024-09-11T22:13:17.037964+0000","last_peered":"2024-09-11T22:13:17.037964+0000","last_clean":"2024-09-11T22:13:17.037964+0000","last_became_active":"2024-09-11T22:12:02.771296+0000","last_became_peered":"2024-09-11T22:12:02.771296+0000","last_unstale":"2024-09-11T22:13:17.037964+0000","last_undegraded":"2024-09-11T22:13:17.037964+0000","last_fullsized":"2024-09-11T22:13:17.037964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:05.144166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776881+0000","last_change":"2024-09-11T22:12:00.744431+0000","last_active":"2024-09-11T22:13:16.776881+0000","last_peered":"2024-09-11T22:13:16.776881+0000","last_clean":"2024-09-11T22:13:16.776881+0000","last_became_active":"2024-09-11T22:12:00.744121+0000","last_became_peered":"2024-09-11T22:12:00.744121+0000","last_unstale":"2024-09-11T22:13:16.776881+0000","last_undegraded":"2024-09-11T22:13:16.776881+0000","last_fullsized":"2024-09-11T22:13:16.776881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:16.832914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779230+0000","last_change":"2024-09-11T22:10:51.241028+0000","last_active":"2024-09-11T22:13:16.779230+0000","last_peered":"2024-09-11T22:13:16.779230+0000","last_clean":"2024-09-11T22:13:16.779230+0000","last_became_active":"2024-09-11T22:10:51.240533+0000","last_became_peered":"2024-09-11T22:10:51.240533+0000","last_unstale":"2024-09-11T22:13:16.779230+0000","last_undegraded":"2024-09-11T22:13:16.779230+0000","last_fullsized":"2024-09-11T22:13:16.779230+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:07:30.402427+0000","last_clean_scrub_stamp":"2024-09-11T22:07:30.402427+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:59.522909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779952+0000","last_change":"2024-09-11T22:12:09.815386+0000","last_active":"2024-09-11T22:13:16.779952+0000","last_peered":"2024-09-11T22:13:16.779952+0000","last_clean":"2024-09-11T22:13:16.779952+0000","last_became_active":"2024-09-11T22:12:09.815004+0000","last_became_peered":"2024-09-11T22:12:09.815004+0000","last_unstale":"2024-09-11T22:13:16.779952+0000","last_undegraded":"2024-09-11T22:13:16.779952+0000","last_fullsized":"2024-09-11T22:13:16.779952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:22.247309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780179+0000","last_change":"2024-09-11T22:12:02.774139+0000","last_active":"2024-09-11T22:13:16.780179+0000","last_peered":"2024-09-11T22:13:16.780179+0000","last_clean":"2024-09-11T22:13:16.780179+0000","last_became_active":"2024-09-11T22:12:02.773785+0000","last_became_peered":"2024-09-11T22:12:02.773785+0000","last_unstale":"2024-09-11T22:13:16.780179+0000","last_undegraded":"2024-09-11T22:13:16.780179+0000","last_fullsized":"2024-09-11T22:13:16.780179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:53.160278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":95,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:32.246259+0000","last_change":"2024-09-11T22:12:00.818813+0000","last_active":"2024-09-11T22:13:32.246259+0000","last_peered":"2024-09-11T22:13:32.246259+0000","last_clean":"2024-09-11T22:13:32.246259+0000","last_became_active":"2024-09-11T22:12:00.817574+0000","last_became_peered":"2024-09-11T22:12:00.817574+0000","last_unstale":"2024-09-11T22:13:32.246259+0000","last_undegraded":"2024-09-11T22:13:32.246259+0000","last_fullsized":"2024-09-11T22:13:32.246259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:52.429428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784870+0000","last_change":"2024-09-11T22:12:09.819009+0000","last_active":"2024-09-11T22:13:16.784870+0000","last_peered":"2024-09-11T22:13:16.784870+0000","last_clean":"2024-09-11T22:13:16.784870+0000","last_became_active":"2024-09-11T22:12:09.818513+0000","last_became_peered":"2024-09-11T22:12:09.818513+0000","last_unstale":"2024-09-11T22:13:16.784870+0000","last_undegraded":"2024-09-11T22:13:16.784870+0000","last_fullsized":"2024-09-11T22:13:16.784870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:34.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779778+0000","last_change":"2024-09-11T22:12:02.776027+0000","last_active":"2024-09-11T22:13:16.779778+0000","last_peered":"2024-09-11T22:13:16.779778+0000","last_clean":"2024-09-11T22:13:16.779778+0000","last_became_active":"2024-09-11T22:12:02.775485+0000","last_became_peered":"2024-09-11T22:12:02.775485+0000","last_unstale":"2024-09-11T22:13:16.779778+0000","last_undegraded":"2024-09-11T22:13:16.779778+0000","last_fullsized":"2024-09-11T22:13:16.779778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:02.801205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784564+0000","last_change":"2024-09-11T22:12:00.747291+0000","last_active":"2024-09-11T22:13:16.784564+0000","last_peered":"2024-09-11T22:13:16.784564+0000","last_clean":"2024-09-11T22:13:16.784564+0000","last_became_active":"2024-09-11T22:12:00.746910+0000","last_became_peered":"2024-09-11T22:12:00.746910+0000","last_unstale":"2024-09-11T22:13:16.784564+0000","last_undegraded":"2024-09-11T22:13:16.784564+0000","last_fullsized":"2024-09-11T22:13:16.784564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:23.993924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786426+0000","last_change":"2024-09-11T22:13:16.786426+0000","last_active":"2024-09-11T22:13:16.786426+0000","last_peered":"2024-09-11T22:13:16.786426+0000","last_clean":"2024-09-11T22:13:16.786426+0000","last_became_active":"2024-09-11T22:13:16.785964+0000","last_became_peered":"2024-09-11T22:13:16.785964+0000","last_unstale":"2024-09-11T22:13:16.786426+0000","last_undegraded":"2024-09-11T22:13:16.786426+0000","last_fullsized":"2024-09-11T22:13:16.786426+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:48.265620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779815+0000","last_change":"2024-09-11T22:12:02.766451+0000","last_active":"2024-09-11T22:13:16.779815+0000","last_peered":"2024-09-11T22:13:16.779815+0000","last_clean":"2024-09-11T22:13:16.779815+0000","last_became_active":"2024-09-11T22:12:02.765909+0000","last_became_peered":"2024-09-11T22:12:02.765909+0000","last_unstale":"2024-09-11T22:13:16.779815+0000","last_undegraded":"2024-09-11T22:13:16.779815+0000","last_fullsized":"2024-09-11T22:13:16.779815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:34.099196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038072+0000","last_change":"2024-09-11T22:12:00.748989+0000","last_active":"2024-09-11T22:13:17.038072+0000","last_peered":"2024-09-11T22:13:17.038072+0000","last_clean":"2024-09-11T22:13:17.038072+0000","last_became_active":"2024-09-11T22:12:00.748653+0000","last_became_peered":"2024-09-11T22:12:00.748653+0000","last_unstale":"2024-09-11T22:13:17.038072+0000","last_undegraded":"2024-09-11T22:13:17.038072+0000","last_fullsized":"2024-09-11T22:13:17.038072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:53.234719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782374+0000","last_change":"2024-09-11T22:12:09.818187+0000","last_active":"2024-09-11T22:13:16.782374+0000","last_peered":"2024-09-11T22:13:16.782374+0000","last_clean":"2024-09-11T22:13:16.782374+0000","last_became_active":"2024-09-11T22:12:09.817355+0000","last_became_peered":"2024-09-11T22:12:09.817355+0000","last_unstale":"2024-09-11T22:13:16.782374+0000","last_undegraded":"2024-09-11T22:13:16.782374+0000","last_fullsized":"2024-09-11T22:13:16.782374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:57.355716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779302+0000","last_change":"2024-09-11T22:12:02.774309+0000","last_active":"2024-09-11T22:13:16.779302+0000","last_peered":"2024-09-11T22:13:16.779302+0000","last_clean":"2024-09-11T22:13:16.779302+0000","last_became_active":"2024-09-11T22:12:02.774004+0000","last_became_peered":"2024-09-11T22:12:02.774004+0000","last_unstale":"2024-09-11T22:13:16.779302+0000","last_undegraded":"2024-09-11T22:13:16.779302+0000","last_fullsized":"2024-09-11T22:13:16.779302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:03.107880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.791515+0000","last_change":"2024-09-11T22:13:16.791017+0000","last_active":"2024-09-11T22:13:16.791515+0000","last_peered":"2024-09-11T22:13:16.791515+0000","last_clean":"2024-09-11T22:13:16.791515+0000","last_became_active":"2024-09-11T22:13:16.784755+0000","last_became_peered":"2024-09-11T22:13:16.784755+0000","last_unstale":"2024-09-11T22:13:16.791515+0000","last_undegraded":"2024-09-11T22:13:16.791515+0000","last_fullsized":"2024-09-11T22:13:16.791515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:53.317504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.783171+0000","last_change":"2024-09-11T22:12:09.821945+0000","last_active":"2024-09-11T22:13:16.783171+0000","last_peered":"2024-09-11T22:13:16.783171+0000","last_clean":"2024-09-11T22:13:16.783171+0000","last_became_active":"2024-09-11T22:12:09.821537+0000","last_became_peered":"2024-09-11T22:12:09.821537+0000","last_unstale":"2024-09-11T22:13:16.783171+0000","last_undegraded":"2024-09-11T22:13:16.783171+0000","last_fullsized":"2024-09-11T22:13:16.783171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:18.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.783348+0000","last_change":"2024-09-11T22:12:02.768511+0000","last_active":"2024-09-11T22:13:16.783348+0000","last_peered":"2024-09-11T22:13:16.783348+0000","last_clean":"2024-09-11T22:13:16.783348+0000","last_became_active":"2024-09-11T22:12:02.768060+0000","last_became_peered":"2024-09-11T22:12:02.768060+0000","last_unstale":"2024-09-11T22:13:16.783348+0000","last_undegraded":"2024-09-11T22:13:16.783348+0000","last_fullsized":"2024-09-11T22:13:16.783348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:26.417066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776143+0000","last_change":"2024-09-11T22:12:00.817941+0000","last_active":"2024-09-11T22:13:16.776143+0000","last_peered":"2024-09-11T22:13:16.776143+0000","last_clean":"2024-09-11T22:13:16.776143+0000","last_became_active":"2024-09-11T22:12:00.817606+0000","last_became_peered":"2024-09-11T22:12:00.817606+0000","last_unstale":"2024-09-11T22:13:16.776143+0000","last_undegraded":"2024-09-11T22:13:16.776143+0000","last_fullsized":"2024-09-11T22:13:16.776143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:44.212961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038195+0000","last_change":"2024-09-11T22:12:09.812516+0000","last_active":"2024-09-11T22:13:17.038195+0000","last_peered":"2024-09-11T22:13:17.038195+0000","last_clean":"2024-09-11T22:13:17.038195+0000","last_became_active":"2024-09-11T22:12:09.812217+0000","last_became_peered":"2024-09-11T22:12:09.812217+0000","last_unstale":"2024-09-11T22:13:17.038195+0000","last_undegraded":"2024-09-11T22:13:17.038195+0000","last_fullsized":"2024-09-11T22:13:17.038195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:36.568803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778984+0000","last_change":"2024-09-11T22:12:02.775797+0000","last_active":"2024-09-11T22:13:16.778984+0000","last_peered":"2024-09-11T22:13:16.778984+0000","last_clean":"2024-09-11T22:13:16.778984+0000","last_became_active":"2024-09-11T22:12:02.775474+0000","last_became_peered":"2024-09-11T22:12:02.775474+0000","last_unstale":"2024-09-11T22:13:16.778984+0000","last_undegraded":"2024-09-11T22:13:16.778984+0000","last_fullsized":"2024-09-11T22:13:16.778984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:25.446633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":96,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:31.223463+0000","last_change":"2024-09-11T22:12:00.744237+0000","last_active":"2024-09-11T22:13:31.223463+0000","last_peered":"2024-09-11T22:13:31.223463+0000","last_clean":"2024-09-11T22:13:31.223463+0000","last_became_active":"2024-09-11T22:12:00.743783+0000","last_became_peered":"2024-09-11T22:12:00.743783+0000","last_unstale":"2024-09-11T22:13:31.223463+0000","last_undegraded":"2024-09-11T22:13:31.223463+0000","last_fullsized":"2024-09-11T22:13:31.223463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:53.046685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775794+0000","last_change":"2024-09-11T22:12:09.815279+0000","last_active":"2024-09-11T22:13:16.775794+0000","last_peered":"2024-09-11T22:13:16.775794+0000","last_clean":"2024-09-11T22:13:16.775794+0000","last_became_active":"2024-09-11T22:12:09.814447+0000","last_became_peered":"2024-09-11T22:12:09.814447+0000","last_unstale":"2024-09-11T22:13:16.775794+0000","last_undegraded":"2024-09-11T22:13:16.775794+0000","last_fullsized":"2024-09-11T22:13:16.775794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.289292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.039259+0000","last_change":"2024-09-11T22:13:17.039259+0000","last_active":"2024-09-11T22:13:17.039259+0000","last_peered":"2024-09-11T22:13:17.039259+0000","last_clean":"2024-09-11T22:13:17.039259+0000","last_became_active":"2024-09-11T22:13:17.038840+0000","last_became_peered":"2024-09-11T22:13:17.038840+0000","last_unstale":"2024-09-11T22:13:17.039259+0000","last_undegraded":"2024-09-11T22:13:17.039259+0000","last_fullsized":"2024-09-11T22:13:17.039259+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:55.538975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.783817+0000","last_change":"2024-09-11T22:12:00.749008+0000","last_active":"2024-09-11T22:13:16.783817+0000","last_peered":"2024-09-11T22:13:16.783817+0000","last_clean":"2024-09-11T22:13:16.783817+0000","last_became_active":"2024-09-11T22:12:00.748642+0000","last_became_peered":"2024-09-11T22:12:00.748642+0000","last_unstale":"2024-09-11T22:13:16.783817+0000","last_undegraded":"2024-09-11T22:13:16.783817+0000","last_fullsized":"2024-09-11T22:13:16.783817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:34.763101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778735+0000","last_change":"2024-09-11T22:12:09.816925+0000","last_active":"2024-09-11T22:13:16.778735+0000","last_peered":"2024-09-11T22:13:16.778735+0000","last_clean":"2024-09-11T22:13:16.778735+0000","last_became_active":"2024-09-11T22:12:09.816434+0000","last_became_peered":"2024-09-11T22:12:09.816434+0000","last_unstale":"2024-09-11T22:13:16.778735+0000","last_undegraded":"2024-09-11T22:13:16.778735+0000","last_fullsized":"2024-09-11T22:13:16.778735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:50.127861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037551+0000","last_change":"2024-09-11T22:12:02.761721+0000","last_active":"2024-09-11T22:13:17.037551+0000","last_peered":"2024-09-11T22:13:17.037551+0000","last_clean":"2024-09-11T22:13:17.037551+0000","last_became_active":"2024-09-11T22:12:02.761448+0000","last_became_peered":"2024-09-11T22:12:02.761448+0000","last_unstale":"2024-09-11T22:13:17.037551+0000","last_undegraded":"2024-09-11T22:13:17.037551+0000","last_fullsized":"2024-09-11T22:13:17.037551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:51.401729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781599+0000","last_change":"2024-09-11T22:12:00.751110+0000","last_active":"2024-09-11T22:13:16.781599+0000","last_peered":"2024-09-11T22:13:16.781599+0000","last_clean":"2024-09-11T22:13:16.781599+0000","last_became_active":"2024-09-11T22:12:00.749431+0000","last_became_peered":"2024-09-11T22:12:00.749431+0000","last_unstale":"2024-09-11T22:13:16.781599+0000","last_undegraded":"2024-09-11T22:13:16.781599+0000","last_fullsized":"2024-09-11T22:13:16.781599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:27.774865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784188+0000","last_change":"2024-09-11T22:12:09.820852+0000","last_active":"2024-09-11T22:13:16.784188+0000","last_peered":"2024-09-11T22:13:16.784188+0000","last_clean":"2024-09-11T22:13:16.784188+0000","last_became_active":"2024-09-11T22:12:09.820263+0000","last_became_peered":"2024-09-11T22:12:09.820263+0000","last_unstale":"2024-09-11T22:13:16.784188+0000","last_undegraded":"2024-09-11T22:13:16.784188+0000","last_fullsized":"2024-09-11T22:13:16.784188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:09.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776776+0000","last_change":"2024-09-11T22:12:02.770819+0000","last_active":"2024-09-11T22:13:16.776776+0000","last_peered":"2024-09-11T22:13:16.776776+0000","last_clean":"2024-09-11T22:13:16.776776+0000","last_became_active":"2024-09-11T22:12:02.770502+0000","last_became_peered":"2024-09-11T22:12:02.770502+0000","last_unstale":"2024-09-11T22:13:16.776776+0000","last_undegraded":"2024-09-11T22:13:16.776776+0000","last_fullsized":"2024-09-11T22:13:16.776776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:10.345880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":196,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:28.805541+0000","last_change":"2024-09-11T22:13:16.798183+0000","last_active":"2024-09-11T22:13:28.805541+0000","last_peered":"2024-09-11T22:13:28.805541+0000","last_clean":"2024-09-11T22:13:28.805541+0000","last_became_active":"2024-09-11T22:13:16.787422+0000","last_became_peered":"2024-09-11T22:13:16.787422+0000","last_unstale":"2024-09-11T22:13:28.805541+0000","last_undegraded":"2024-09-11T22:13:28.805541+0000","last_fullsized":"2024-09-11T22:13:28.805541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:15.849975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63484,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":112,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":63484,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775334+0000","last_change":"2024-09-11T22:12:09.819643+0000","last_active":"2024-09-11T22:13:16.775334+0000","last_peered":"2024-09-11T22:13:16.775334+0000","last_clean":"2024-09-11T22:13:16.775334+0000","last_became_active":"2024-09-11T22:12:09.819144+0000","last_became_peered":"2024-09-11T22:12:09.819144+0000","last_unstale":"2024-09-11T22:13:16.775334+0000","last_undegraded":"2024-09-11T22:13:16.775334+0000","last_fullsized":"2024-09-11T22:13:16.775334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.006995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784964+0000","last_change":"2024-09-11T22:12:02.762553+0000","last_active":"2024-09-11T22:13:16.784964+0000","last_peered":"2024-09-11T22:13:16.784964+0000","last_clean":"2024-09-11T22:13:16.784964+0000","last_became_active":"2024-09-11T22:12:02.762197+0000","last_became_peered":"2024-09-11T22:12:02.762197+0000","last_unstale":"2024-09-11T22:13:16.784964+0000","last_undegraded":"2024-09-11T22:13:16.784964+0000","last_fullsized":"2024-09-11T22:13:16.784964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:07.188949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784508+0000","last_change":"2024-09-11T22:12:00.817959+0000","last_active":"2024-09-11T22:13:16.784508+0000","last_peered":"2024-09-11T22:13:16.784508+0000","last_clean":"2024-09-11T22:13:16.784508+0000","last_became_active":"2024-09-11T22:12:00.817615+0000","last_became_peered":"2024-09-11T22:12:00.817615+0000","last_unstale":"2024-09-11T22:13:16.784508+0000","last_undegraded":"2024-09-11T22:13:16.784508+0000","last_fullsized":"2024-09-11T22:13:16.784508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:33.671580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786618+0000","last_change":"2024-09-11T22:13:16.786618+0000","last_active":"2024-09-11T22:13:16.786618+0000","last_peered":"2024-09-11T22:13:16.786618+0000","last_clean":"2024-09-11T22:13:16.786618+0000","last_became_active":"2024-09-11T22:13:16.786052+0000","last_became_peered":"2024-09-11T22:13:16.786052+0000","last_unstale":"2024-09-11T22:13:16.786618+0000","last_undegraded":"2024-09-11T22:13:16.786618+0000","last_fullsized":"2024-09-11T22:13:16.786618+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:19.092080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778314+0000","last_change":"2024-09-11T22:12:02.773369+0000","last_active":"2024-09-11T22:13:16.778314+0000","last_peered":"2024-09-11T22:13:16.778314+0000","last_clean":"2024-09-11T22:13:16.778314+0000","last_became_active":"2024-09-11T22:12:02.773046+0000","last_became_peered":"2024-09-11T22:12:02.773046+0000","last_unstale":"2024-09-11T22:13:16.778314+0000","last_undegraded":"2024-09-11T22:13:16.778314+0000","last_fullsized":"2024-09-11T22:13:16.778314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:04.993273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780135+0000","last_change":"2024-09-11T22:12:00.751696+0000","last_active":"2024-09-11T22:13:16.780135+0000","last_peered":"2024-09-11T22:13:16.780135+0000","last_clean":"2024-09-11T22:13:16.780135+0000","last_became_active":"2024-09-11T22:12:00.751362+0000","last_became_peered":"2024-09-11T22:12:00.751362+0000","last_unstale":"2024-09-11T22:13:16.780135+0000","last_undegraded":"2024-09-11T22:13:16.780135+0000","last_fullsized":"2024-09-11T22:13:16.780135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:19.372942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775149+0000","last_change":"2024-09-11T22:12:09.814219+0000","last_active":"2024-09-11T22:13:16.775149+0000","last_peered":"2024-09-11T22:13:16.775149+0000","last_clean":"2024-09-11T22:13:16.775149+0000","last_became_active":"2024-09-11T22:12:09.813817+0000","last_became_peered":"2024-09-11T22:12:09.813817+0000","last_unstale":"2024-09-11T22:13:16.775149+0000","last_undegraded":"2024-09-11T22:13:16.775149+0000","last_fullsized":"2024-09-11T22:13:16.775149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:36.473171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.785654+0000","last_change":"2024-09-11T22:12:02.769739+0000","last_active":"2024-09-11T22:13:16.785654+0000","last_peered":"2024-09-11T22:13:16.785654+0000","last_clean":"2024-09-11T22:13:16.785654+0000","last_became_active":"2024-09-11T22:12:02.769118+0000","last_became_peered":"2024-09-11T22:12:02.769118+0000","last_unstale":"2024-09-11T22:13:16.785654+0000","last_undegraded":"2024-09-11T22:13:16.785654+0000","last_fullsized":"2024-09-11T22:13:16.785654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:01.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:29.224282+0000","last_change":"2024-09-11T22:12:00.748294+0000","last_active":"2024-09-11T22:13:29.224282+0000","last_peered":"2024-09-11T22:13:29.224282+0000","last_clean":"2024-09-11T22:13:29.224282+0000","last_became_active":"2024-09-11T22:12:00.748000+0000","last_became_peered":"2024-09-11T22:12:00.748000+0000","last_unstale":"2024-09-11T22:13:29.224282+0000","last_undegraded":"2024-09-11T22:13:29.224282+0000","last_fullsized":"2024-09-11T22:13:29.224282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:08.629756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037471+0000","last_change":"2024-09-11T22:12:09.820051+0000","last_active":"2024-09-11T22:13:17.037471+0000","last_peered":"2024-09-11T22:13:17.037471+0000","last_clean":"2024-09-11T22:13:17.037471+0000","last_became_active":"2024-09-11T22:12:09.819655+0000","last_became_peered":"2024-09-11T22:12:09.819655+0000","last_unstale":"2024-09-11T22:13:17.037471+0000","last_undegraded":"2024-09-11T22:13:17.037471+0000","last_fullsized":"2024-09-11T22:13:17.037471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:26.560012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777784+0000","last_change":"2024-09-11T22:12:09.812991+0000","last_active":"2024-09-11T22:13:16.777784+0000","last_peered":"2024-09-11T22:13:16.777784+0000","last_clean":"2024-09-11T22:13:16.777784+0000","last_became_active":"2024-09-11T22:12:09.812602+0000","last_became_peered":"2024-09-11T22:12:09.812602+0000","last_unstale":"2024-09-11T22:13:16.777784+0000","last_undegraded":"2024-09-11T22:13:16.777784+0000","last_fullsized":"2024-09-11T22:13:16.777784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:21.198192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775258+0000","last_change":"2024-09-11T22:12:02.760545+0000","last_active":"2024-09-11T22:13:16.775258+0000","last_peered":"2024-09-11T22:13:16.775258+0000","last_clean":"2024-09-11T22:13:16.775258+0000","last_became_active":"2024-09-11T22:12:02.760061+0000","last_became_peered":"2024-09-11T22:12:02.760061+0000","last_unstale":"2024-09-11T22:13:16.775258+0000","last_undegraded":"2024-09-11T22:13:16.775258+0000","last_fullsized":"2024-09-11T22:13:16.775258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:37.000552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782907+0000","last_change":"2024-09-11T22:12:00.750643+0000","last_active":"2024-09-11T22:13:16.782907+0000","last_peered":"2024-09-11T22:13:16.782907+0000","last_clean":"2024-09-11T22:13:16.782907+0000","last_became_active":"2024-09-11T22:12:00.750129+0000","last_became_peered":"2024-09-11T22:12:00.750129+0000","last_unstale":"2024-09-11T22:13:16.782907+0000","last_undegraded":"2024-09-11T22:13:16.782907+0000","last_fullsized":"2024-09-11T22:13:16.782907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:25.519952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786013+0000","last_change":"2024-09-11T22:13:16.786013+0000","last_active":"2024-09-11T22:13:16.786013+0000","last_peered":"2024-09-11T22:13:16.786013+0000","last_clean":"2024-09-11T22:13:16.786013+0000","last_became_active":"2024-09-11T22:13:16.785356+0000","last_became_peered":"2024-09-11T22:13:16.785356+0000","last_unstale":"2024-09-11T22:13:16.786013+0000","last_undegraded":"2024-09-11T22:13:16.786013+0000","last_fullsized":"2024-09-11T22:13:16.786013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:51:39.411078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037491+0000","last_change":"2024-09-11T22:12:02.766428+0000","last_active":"2024-09-11T22:13:17.037491+0000","last_peered":"2024-09-11T22:13:17.037491+0000","last_clean":"2024-09-11T22:13:17.037491+0000","last_became_active":"2024-09-11T22:12:02.766112+0000","last_became_peered":"2024-09-11T22:12:02.766112+0000","last_unstale":"2024-09-11T22:13:17.037491+0000","last_undegraded":"2024-09-11T22:13:17.037491+0000","last_fullsized":"2024-09-11T22:13:17.037491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:24.701996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037817+0000","last_change":"2024-09-11T22:12:00.743654+0000","last_active":"2024-09-11T22:13:17.037817+0000","last_peered":"2024-09-11T22:13:17.037817+0000","last_clean":"2024-09-11T22:13:17.037817+0000","last_became_active":"2024-09-11T22:12:00.743216+0000","last_became_peered":"2024-09-11T22:12:00.743216+0000","last_unstale":"2024-09-11T22:13:17.037817+0000","last_undegraded":"2024-09-11T22:13:17.037817+0000","last_fullsized":"2024-09-11T22:13:17.037817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.843451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781136+0000","last_change":"2024-09-11T22:12:00.747738+0000","last_active":"2024-09-11T22:13:16.781136+0000","last_peered":"2024-09-11T22:13:16.781136+0000","last_clean":"2024-09-11T22:13:16.781136+0000","last_became_active":"2024-09-11T22:12:00.747424+0000","last_became_peered":"2024-09-11T22:12:00.747424+0000","last_unstale":"2024-09-11T22:13:16.781136+0000","last_undegraded":"2024-09-11T22:13:16.781136+0000","last_fullsized":"2024-09-11T22:13:16.781136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:30.097802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784023+0000","last_change":"2024-09-11T22:12:02.770033+0000","last_active":"2024-09-11T22:13:16.784023+0000","last_peered":"2024-09-11T22:13:16.784023+0000","last_clean":"2024-09-11T22:13:16.784023+0000","last_became_active":"2024-09-11T22:12:02.769597+0000","last_became_peered":"2024-09-11T22:12:02.769597+0000","last_unstale":"2024-09-11T22:13:16.784023+0000","last_undegraded":"2024-09-11T22:13:16.784023+0000","last_fullsized":"2024-09-11T22:13:16.784023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:35.447477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780438+0000","last_change":"2024-09-11T22:12:09.816652+0000","last_active":"2024-09-11T22:13:16.780438+0000","last_peered":"2024-09-11T22:13:16.780438+0000","last_clean":"2024-09-11T22:13:16.780438+0000","last_became_active":"2024-09-11T22:12:09.816259+0000","last_became_peered":"2024-09-11T22:12:09.816259+0000","last_unstale":"2024-09-11T22:13:16.780438+0000","last_undegraded":"2024-09-11T22:13:16.780438+0000","last_fullsized":"2024-09-11T22:13:16.780438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:09.818890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":95,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:32.229204+0000","last_change":"2024-09-11T22:12:00.752118+0000","last_active":"2024-09-11T22:13:32.229204+0000","last_peered":"2024-09-11T22:13:32.229204+0000","last_clean":"2024-09-11T22:13:32.229204+0000","last_became_active":"2024-09-11T22:12:00.751757+0000","last_became_peered":"2024-09-11T22:12:00.751757+0000","last_unstale":"2024-09-11T22:13:32.229204+0000","last_undegraded":"2024-09-11T22:13:32.229204+0000","last_fullsized":"2024-09-11T22:13:32.229204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:11:59.724443+0000","last_clean_scrub_stamp":"2024-09-11T22:11:59.724443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:28.139391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776278+0000","last_change":"2024-09-11T22:12:02.772836+0000","last_active":"2024-09-11T22:13:16.776278+0000","last_peered":"2024-09-11T22:13:16.776278+0000","last_clean":"2024-09-11T22:13:16.776278+0000","last_became_active":"2024-09-11T22:12:02.772528+0000","last_became_peered":"2024-09-11T22:12:02.772528+0000","last_unstale":"2024-09-11T22:13:16.776278+0000","last_undegraded":"2024-09-11T22:13:16.776278+0000","last_fullsized":"2024-09-11T22:13:16.776278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:05.148091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780067+0000","last_change":"2024-09-11T22:12:09.820138+0000","last_active":"2024-09-11T22:13:16.780067+0000","last_peered":"2024-09-11T22:13:16.780067+0000","last_clean":"2024-09-11T22:13:16.780067+0000","last_became_active":"2024-09-11T22:12:09.819428+0000","last_became_peered":"2024-09-11T22:12:09.819428+0000","last_unstale":"2024-09-11T22:13:16.780067+0000","last_undegraded":"2024-09-11T22:13:16.780067+0000","last_fullsized":"2024-09-11T22:13:16.780067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:08.783149+0000","last_clean_scrub_stamp":"2024-09-11T22:12:08.783149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:10.958913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778264+0000","last_change":"2024-09-11T22:12:02.772732+0000","last_active":"2024-09-11T22:13:16.778264+0000","last_peered":"2024-09-11T22:13:16.778264+0000","last_clean":"2024-09-11T22:13:16.778264+0000","last_became_active":"2024-09-11T22:12:02.772407+0000","last_became_peered":"2024-09-11T22:12:02.772407+0000","last_unstale":"2024-09-11T22:13:16.778264+0000","last_undegraded":"2024-09-11T22:13:16.778264+0000","last_fullsized":"2024-09-11T22:13:16.778264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:24.134356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781026+0000","last_change":"2024-09-11T22:12:02.775825+0000","last_active":"2024-09-11T22:13:16.781026+0000","last_peered":"2024-09-11T22:13:16.781026+0000","last_clean":"2024-09-11T22:13:16.781026+0000","last_became_active":"2024-09-11T22:12:02.775471+0000","last_became_peered":"2024-09-11T22:12:02.775471+0000","last_unstale":"2024-09-11T22:13:16.781026+0000","last_undegraded":"2024-09-11T22:13:16.781026+0000","last_fullsized":"2024-09-11T22:13:16.781026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:46.333390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779899+0000","last_change":"2024-09-11T22:12:02.764570+0000","last_active":"2024-09-11T22:13:16.779899+0000","last_peered":"2024-09-11T22:13:16.779899+0000","last_clean":"2024-09-11T22:13:16.779899+0000","last_became_active":"2024-09-11T22:12:02.764005+0000","last_became_peered":"2024-09-11T22:12:02.764005+0000","last_unstale":"2024-09-11T22:13:16.779899+0000","last_undegraded":"2024-09-11T22:13:16.779899+0000","last_fullsized":"2024-09-11T22:13:16.779899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:32.169886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777496+0000","last_change":"2024-09-11T22:12:02.769156+0000","last_active":"2024-09-11T22:13:16.777496+0000","last_peered":"2024-09-11T22:13:16.777496+0000","last_clean":"2024-09-11T22:13:16.777496+0000","last_became_active":"2024-09-11T22:12:02.768795+0000","last_became_peered":"2024-09-11T22:12:02.768795+0000","last_unstale":"2024-09-11T22:13:16.777496+0000","last_undegraded":"2024-09-11T22:13:16.777496+0000","last_fullsized":"2024-09-11T22:13:16.777496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:25.506739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037816+0000","last_change":"2024-09-11T22:12:02.763102+0000","last_active":"2024-09-11T22:13:17.037816+0000","last_peered":"2024-09-11T22:13:17.037816+0000","last_clean":"2024-09-11T22:13:17.037816+0000","last_became_active":"2024-09-11T22:12:02.762687+0000","last_became_peered":"2024-09-11T22:12:02.762687+0000","last_unstale":"2024-09-11T22:13:17.037816+0000","last_undegraded":"2024-09-11T22:13:17.037816+0000","last_fullsized":"2024-09-11T22:13:17.037816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:05.069744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.779122+0000","last_change":"2024-09-11T22:12:02.776229+0000","last_active":"2024-09-11T22:13:16.779122+0000","last_peered":"2024-09-11T22:13:16.779122+0000","last_clean":"2024-09-11T22:13:16.779122+0000","last_became_active":"2024-09-11T22:12:02.775875+0000","last_became_peered":"2024-09-11T22:12:02.775875+0000","last_unstale":"2024-09-11T22:13:16.779122+0000","last_undegraded":"2024-09-11T22:13:16.779122+0000","last_fullsized":"2024-09-11T22:13:16.779122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:46.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781971+0000","last_change":"2024-09-11T22:12:02.766783+0000","last_active":"2024-09-11T22:13:16.781971+0000","last_peered":"2024-09-11T22:13:16.781971+0000","last_clean":"2024-09-11T22:13:16.781971+0000","last_became_active":"2024-09-11T22:12:02.766280+0000","last_became_peered":"2024-09-11T22:12:02.766280+0000","last_unstale":"2024-09-11T22:13:16.781971+0000","last_undegraded":"2024-09-11T22:13:16.781971+0000","last_fullsized":"2024-09-11T22:13:16.781971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:12.657682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038187+0000","last_change":"2024-09-11T22:12:02.759008+0000","last_active":"2024-09-11T22:13:17.038187+0000","last_peered":"2024-09-11T22:13:17.038187+0000","last_clean":"2024-09-11T22:13:17.038187+0000","last_became_active":"2024-09-11T22:12:02.758734+0000","last_became_peered":"2024-09-11T22:12:02.758734+0000","last_unstale":"2024-09-11T22:13:17.038187+0000","last_undegraded":"2024-09-11T22:13:17.038187+0000","last_fullsized":"2024-09-11T22:13:17.038187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:37.729891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782612+0000","last_change":"2024-09-11T22:12:02.777260+0000","last_active":"2024-09-11T22:13:16.782612+0000","last_peered":"2024-09-11T22:13:16.782612+0000","last_clean":"2024-09-11T22:13:16.782612+0000","last_became_active":"2024-09-11T22:12:02.775486+0000","last_became_peered":"2024-09-11T22:12:02.775486+0000","last_unstale":"2024-09-11T22:13:16.782612+0000","last_undegraded":"2024-09-11T22:13:16.782612+0000","last_fullsized":"2024-09-11T22:13:16.782612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:41.099229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037635+0000","last_change":"2024-09-11T22:12:02.762025+0000","last_active":"2024-09-11T22:13:17.037635+0000","last_peered":"2024-09-11T22:13:17.037635+0000","last_clean":"2024-09-11T22:13:17.037635+0000","last_became_active":"2024-09-11T22:12:02.761751+0000","last_became_peered":"2024-09-11T22:12:02.761751+0000","last_unstale":"2024-09-11T22:13:17.037635+0000","last_undegraded":"2024-09-11T22:13:17.037635+0000","last_fullsized":"2024-09-11T22:13:17.037635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:45.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.785090+0000","last_change":"2024-09-11T22:12:02.765001+0000","last_active":"2024-09-11T22:13:16.785090+0000","last_peered":"2024-09-11T22:13:16.785090+0000","last_clean":"2024-09-11T22:13:16.785090+0000","last_became_active":"2024-09-11T22:12:02.764667+0000","last_became_peered":"2024-09-11T22:12:02.764667+0000","last_unstale":"2024-09-11T22:13:16.785090+0000","last_undegraded":"2024-09-11T22:13:16.785090+0000","last_fullsized":"2024-09-11T22:13:16.785090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:59.074616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038339+0000","last_change":"2024-09-11T22:12:02.762972+0000","last_active":"2024-09-11T22:13:17.038339+0000","last_peered":"2024-09-11T22:13:17.038339+0000","last_clean":"2024-09-11T22:13:17.038339+0000","last_became_active":"2024-09-11T22:12:02.762685+0000","last_became_peered":"2024-09-11T22:12:02.762685+0000","last_unstale":"2024-09-11T22:13:17.038339+0000","last_undegraded":"2024-09-11T22:13:17.038339+0000","last_fullsized":"2024-09-11T22:13:17.038339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:34.278584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.778854+0000","last_change":"2024-09-11T22:12:02.776351+0000","last_active":"2024-09-11T22:13:16.778854+0000","last_peered":"2024-09-11T22:13:16.778854+0000","last_clean":"2024-09-11T22:13:16.778854+0000","last_became_active":"2024-09-11T22:12:02.774827+0000","last_became_peered":"2024-09-11T22:12:02.774827+0000","last_unstale":"2024-09-11T22:13:16.778854+0000","last_undegraded":"2024-09-11T22:13:16.778854+0000","last_fullsized":"2024-09-11T22:13:16.778854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:55.887697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781686+0000","last_change":"2024-09-11T22:12:02.757412+0000","last_active":"2024-09-11T22:13:16.781686+0000","last_peered":"2024-09-11T22:13:16.781686+0000","last_clean":"2024-09-11T22:13:16.781686+0000","last_became_active":"2024-09-11T22:12:02.757013+0000","last_became_peered":"2024-09-11T22:12:02.757013+0000","last_unstale":"2024-09-11T22:13:16.781686+0000","last_undegraded":"2024-09-11T22:13:16.781686+0000","last_fullsized":"2024-09-11T22:13:16.781686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:21.468082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.775902+0000","last_change":"2024-09-11T22:12:02.765943+0000","last_active":"2024-09-11T22:13:16.775902+0000","last_peered":"2024-09-11T22:13:16.775902+0000","last_clean":"2024-09-11T22:13:16.775902+0000","last_became_active":"2024-09-11T22:12:02.765617+0000","last_became_peered":"2024-09-11T22:12:02.765617+0000","last_unstale":"2024-09-11T22:13:16.775902+0000","last_undegraded":"2024-09-11T22:13:16.775902+0000","last_fullsized":"2024-09-11T22:13:16.775902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:47.023509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780563+0000","last_change":"2024-09-11T22:12:02.772379+0000","last_active":"2024-09-11T22:13:16.780563+0000","last_peered":"2024-09-11T22:13:16.780563+0000","last_clean":"2024-09-11T22:13:16.780563+0000","last_became_active":"2024-09-11T22:12:02.771561+0000","last_became_peered":"2024-09-11T22:12:02.771561+0000","last_unstale":"2024-09-11T22:13:16.780563+0000","last_undegraded":"2024-09-11T22:13:16.780563+0000","last_fullsized":"2024-09-11T22:13:16.780563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:52.988744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.784295+0000","last_change":"2024-09-11T22:12:02.763489+0000","last_active":"2024-09-11T22:13:16.784295+0000","last_peered":"2024-09-11T22:13:16.784295+0000","last_clean":"2024-09-11T22:13:16.784295+0000","last_became_active":"2024-09-11T22:12:02.763194+0000","last_became_peered":"2024-09-11T22:12:02.763194+0000","last_unstale":"2024-09-11T22:13:16.784295+0000","last_undegraded":"2024-09-11T22:13:16.784295+0000","last_fullsized":"2024-09-11T22:13:16.784295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:15:30.685816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777257+0000","last_change":"2024-09-11T22:12:02.768537+0000","last_active":"2024-09-11T22:13:16.777257+0000","last_peered":"2024-09-11T22:13:16.777257+0000","last_clean":"2024-09-11T22:13:16.777257+0000","last_became_active":"2024-09-11T22:12:02.768219+0000","last_became_peered":"2024-09-11T22:12:02.768219+0000","last_unstale":"2024-09-11T22:13:16.777257+0000","last_undegraded":"2024-09-11T22:13:16.777257+0000","last_fullsized":"2024-09-11T22:13:16.777257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:45:03.177587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777507+0000","last_change":"2024-09-11T22:12:02.761823+0000","last_active":"2024-09-11T22:13:16.777507+0000","last_peered":"2024-09-11T22:13:16.777507+0000","last_clean":"2024-09-11T22:13:16.777507+0000","last_became_active":"2024-09-11T22:12:02.761435+0000","last_became_peered":"2024-09-11T22:12:02.761435+0000","last_unstale":"2024-09-11T22:13:16.777507+0000","last_undegraded":"2024-09-11T22:13:16.777507+0000","last_fullsized":"2024-09-11T22:13:16.777507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:54.535178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.038341+0000","last_change":"2024-09-11T22:12:02.775613+0000","last_active":"2024-09-11T22:13:17.038341+0000","last_peered":"2024-09-11T22:13:17.038341+0000","last_clean":"2024-09-11T22:13:17.038341+0000","last_became_active":"2024-09-11T22:12:02.775383+0000","last_became_peered":"2024-09-11T22:12:02.775383+0000","last_unstale":"2024-09-11T22:13:17.038341+0000","last_undegraded":"2024-09-11T22:13:17.038341+0000","last_fullsized":"2024-09-11T22:13:17.038341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:18.973047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776869+0000","last_change":"2024-09-11T22:12:02.771155+0000","last_active":"2024-09-11T22:13:16.776869+0000","last_peered":"2024-09-11T22:13:16.776869+0000","last_clean":"2024-09-11T22:13:16.776869+0000","last_became_active":"2024-09-11T22:12:02.770278+0000","last_became_peered":"2024-09-11T22:12:02.770278+0000","last_unstale":"2024-09-11T22:13:16.776869+0000","last_undegraded":"2024-09-11T22:13:16.776869+0000","last_fullsized":"2024-09-11T22:13:16.776869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:23.888372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780165+0000","last_change":"2024-09-11T22:12:02.772840+0000","last_active":"2024-09-11T22:13:16.780165+0000","last_peered":"2024-09-11T22:13:16.780165+0000","last_clean":"2024-09-11T22:13:16.780165+0000","last_became_active":"2024-09-11T22:12:02.772384+0000","last_became_peered":"2024-09-11T22:12:02.772384+0000","last_unstale":"2024-09-11T22:13:16.780165+0000","last_undegraded":"2024-09-11T22:13:16.780165+0000","last_fullsized":"2024-09-11T22:13:16.780165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:42.287294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781051+0000","last_change":"2024-09-11T22:12:02.767741+0000","last_active":"2024-09-11T22:13:16.781051+0000","last_peered":"2024-09-11T22:13:16.781051+0000","last_clean":"2024-09-11T22:13:16.781051+0000","last_became_active":"2024-09-11T22:12:02.767234+0000","last_became_peered":"2024-09-11T22:12:02.767234+0000","last_unstale":"2024-09-11T22:13:16.781051+0000","last_undegraded":"2024-09-11T22:13:16.781051+0000","last_fullsized":"2024-09-11T22:13:16.781051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:22.250763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.786025+0000","last_change":"2024-09-11T22:12:02.771265+0000","last_active":"2024-09-11T22:13:16.786025+0000","last_peered":"2024-09-11T22:13:16.786025+0000","last_clean":"2024-09-11T22:13:16.786025+0000","last_became_active":"2024-09-11T22:12:02.770617+0000","last_became_peered":"2024-09-11T22:12:02.770617+0000","last_unstale":"2024-09-11T22:13:16.786025+0000","last_undegraded":"2024-09-11T22:13:16.786025+0000","last_fullsized":"2024-09-11T22:13:16.786025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.390581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:17.037556+0000","last_change":"2024-09-11T22:12:02.770850+0000","last_active":"2024-09-11T22:13:17.037556+0000","last_peered":"2024-09-11T22:13:17.037556+0000","last_clean":"2024-09-11T22:13:17.037556+0000","last_became_active":"2024-09-11T22:12:02.770559+0000","last_became_peered":"2024-09-11T22:12:02.770559+0000","last_unstale":"2024-09-11T22:13:17.037556+0000","last_undegraded":"2024-09-11T22:13:17.037556+0000","last_fullsized":"2024-09-11T22:13:17.037556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:06.865962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780046+0000","last_change":"2024-09-11T22:12:02.775947+0000","last_active":"2024-09-11T22:13:16.780046+0000","last_peered":"2024-09-11T22:13:16.780046+0000","last_clean":"2024-09-11T22:13:16.780046+0000","last_became_active":"2024-09-11T22:12:02.775559+0000","last_became_peered":"2024-09-11T22:12:02.775559+0000","last_unstale":"2024-09-11T22:13:16.780046+0000","last_undegraded":"2024-09-11T22:13:16.780046+0000","last_fullsized":"2024-09-11T22:13:16.780046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:14.642600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.781129+0000","last_change":"2024-09-11T22:12:02.768124+0000","last_active":"2024-09-11T22:13:16.781129+0000","last_peered":"2024-09-11T22:13:16.781129+0000","last_clean":"2024-09-11T22:13:16.781129+0000","last_became_active":"2024-09-11T22:12:02.767804+0000","last_became_peered":"2024-09-11T22:12:02.767804+0000","last_unstale":"2024-09-11T22:13:16.781129+0000","last_undegraded":"2024-09-11T22:13:16.781129+0000","last_fullsized":"2024-09-11T22:13:16.781129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:23.612445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.776413+0000","last_change":"2024-09-11T22:12:02.764086+0000","last_active":"2024-09-11T22:13:16.776413+0000","last_peered":"2024-09-11T22:13:16.776413+0000","last_clean":"2024-09-11T22:13:16.776413+0000","last_became_active":"2024-09-11T22:12:02.763708+0000","last_became_peered":"2024-09-11T22:12:02.763708+0000","last_unstale":"2024-09-11T22:13:16.776413+0000","last_undegraded":"2024-09-11T22:13:16.776413+0000","last_fullsized":"2024-09-11T22:13:16.776413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:06.058314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.782837+0000","last_change":"2024-09-11T22:13:16.782470+0000","last_active":"2024-09-11T22:13:16.782837+0000","last_peered":"2024-09-11T22:13:16.782837+0000","last_clean":"2024-09-11T22:13:16.782837+0000","last_became_active":"2024-09-11T22:13:16.782158+0000","last_became_peered":"2024-09-11T22:13:16.782158+0000","last_unstale":"2024-09-11T22:13:16.782837+0000","last_undegraded":"2024-09-11T22:13:16.782837+0000","last_fullsized":"2024-09-11T22:13:16.782837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:35:37.270023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777779+0000","last_change":"2024-09-11T22:12:02.764702+0000","last_active":"2024-09-11T22:13:16.777779+0000","last_peered":"2024-09-11T22:13:16.777779+0000","last_clean":"2024-09-11T22:13:16.777779+0000","last_became_active":"2024-09-11T22:12:02.764407+0000","last_became_peered":"2024-09-11T22:12:02.764407+0000","last_unstale":"2024-09-11T22:13:16.777779+0000","last_undegraded":"2024-09-11T22:13:16.777779+0000","last_fullsized":"2024-09-11T22:13:16.777779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:01.551823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.777891+0000","last_change":"2024-09-11T22:12:02.762209+0000","last_active":"2024-09-11T22:13:16.777891+0000","last_peered":"2024-09-11T22:13:16.777891+0000","last_clean":"2024-09-11T22:13:16.777891+0000","last_became_active":"2024-09-11T22:12:02.761936+0000","last_became_peered":"2024-09-11T22:12:02.761936+0000","last_unstale":"2024-09-11T22:13:16.777891+0000","last_undegraded":"2024-09-11T22:13:16.777891+0000","last_fullsized":"2024-09-11T22:13:16.777891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:39.667680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T22:13:16.780446+0000","last_change":"2024-09-11T22:12:02.773537+0000","last_active":"2024-09-11T22:13:16.780446+0000","last_peered":"2024-09-11T22:13:16.780446+0000","last_clean":"2024-09-11T22:13:16.780446+0000","last_became_active":"2024-09-11T22:12:02.773145+0000","last_became_peered":"2024-09-11T22:12:02.773145+0000","last_unstale":"2024-09-11T22:13:16.780446+0000","last_undegraded":"2024-09-11T22:13:16.780446+0000","last_fullsized":"2024-09-11T22:13:16.780446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:12:01.737430+0000","last_clean_scrub_stamp":"2024-09-11T22:12:01.737430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:30.058536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70692,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":545,"num_read_kb":341,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":63506,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":212076,"data_compressed":14180,"data_compressed_allocated":20480,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286124,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":510321,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678051,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1155072,"data_stored":1160992,"data_compressed":61883,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":514325,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1024000,"data_stored":510585,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1081344,"data_stored":1107499,"data_compressed":60006,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":506126,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691901,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":1102295,"data_compressed":60006,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855427,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":573684,"data_compressed":34053,"data_compressed_allocated":45056,"data_compressed_original":126976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117064,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":505441,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1024000,"data_stored":506983,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1073152,"data_stored":570640,"data_compressed":25504,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1032192,"data_stored":507189,"data_compressed":23627,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34743,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36379,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65629,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":64577,"data_compressed":10426,"data_compressed_allocated":12288,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65613,"data_compressed":1877,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:13:34.713 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:13:34.713 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T22:13:34.713 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T22:13:34.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T22:13:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:35.032 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:34 smithi092 ceph-mon[31173]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:34 smithi110 ceph-mon[34830]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:35.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:34 smithi153 ceph-mon[35045]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 -- 172.21.15.92:0/1769336075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c100b30 msgr2=0x7f0a0c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 --2- 172.21.15.92:0/1769336075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c100b30 0x7f0a0c100f10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009900 tx=0x7f0a0802a470 comp rx=0 tx=0).stop 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 -- 172.21.15.92:0/1769336075 shutdown_connections 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 --2- 172.21.15.92:0/1769336075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c105f40 0x7f0a0c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 --2- 172.21.15.92:0/1769336075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a0c101450 0x7f0a0c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 --2- 172.21.15.92:0/1769336075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c100b30 0x7f0a0c100f10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 -- 172.21.15.92:0/1769336075 >> 172.21.15.92:0/1769336075 conn(0x7f0a0c0fc7a0 msgr2=0x7f0a0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 -- 172.21.15.92:0/1769336075 shutdown_connections 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.469+0000 7f0a13d34640 1 -- 172.21.15.92:0/1769336075 wait complete. 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 Processor -- start 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 -- start start 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c100b30 0x7f0a0c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c101450 0x7f0a0c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a0c105f40 0x7f0a0c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c118470 con 0x7f0a0c101450 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1182f0 con 0x7f0a0c100b30 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a13d34640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1185f0 con 0x7f0a0c105f40 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a112cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c101450 0x7f0a0c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a112cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c101450 0x7f0a0c1a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51448/0 (socket says 172.21.15.92:51448) 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a112cf640 1 -- 172.21.15.92:0/4445258 learned_addr learned my addr 172.21.15.92:0/4445258 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a11ad0640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c100b30 0x7f0a0c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a122d1640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a0c105f40 0x7f0a0c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a112cf640 1 -- 172.21.15.92:0/4445258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a0c105f40 msgr2=0x7f0a0c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.470+0000 7f0a112cf640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a0c105f40 0x7f0a0c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a112cf640 1 -- 172.21.15.92:0/4445258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c100b30 msgr2=0x7f0a0c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a112cf640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c100b30 0x7f0a0c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a112cf640 1 -- 172.21.15.92:0/4445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c19efd0 con 0x7f0a0c101450 2024-09-11T22:13:35.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a112cf640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c101450 0x7f0a0c1a4bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b3f0 tx=0x7f09fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a02ffd640 1 -- 172.21.15.92:0/4445258 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc015020 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c19f2c0 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a02ffd640 1 -- 172.21.15.92:0/4445258 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09fc011ce0 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a02ffd640 1 -- 172.21.15.92:0/4445258 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc010d80 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.471+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c1ab510 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.472+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0c06b070 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.473+0000 7f0a02ffd640 1 -- 172.21.15.92:0/4445258 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f09fc018020 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.474+0000 7f0a02ffd640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09f80a2c10 0x7f09f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.474+0000 7f0a11ad0640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09f80a2c10 0x7f09f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.474+0000 7f0a02ffd640 1 -- 172.21.15.92:0/4445258 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f09fc0cf2a0 con 0x7f0a0c101450 2024-09-11T22:13:35.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.474+0000 7f0a11ad0640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09f80a2c10 0x7f09f80a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a08031440 tx=0x7f0a08005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:35.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.477+0000 7f0a02ffd640 1 -- 172.21.15.92:0/4445258 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09fc01f020 con 0x7f0a0c101450 2024-09-11T22:13:35.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.678+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0a0c1ab740 con 0x7f0a0c101450 2024-09-11T22:13:35.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.678+0000 7f0a02ffd640 1 -- 172.21.15.92:0/4445258 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f09fc09be50 con 0x7f0a0c101450 2024-09-11T22:13:35.680 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:35.680 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T22:13:35.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09f80a2c10 msgr2=0x7f09f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:35.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09f80a2c10 0x7f09f80a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a08031440 tx=0x7f0a08005b50 comp rx=0 tx=0).stop 2024-09-11T22:13:35.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c101450 msgr2=0x7f0a0c1a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:35.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c101450 0x7f0a0c1a4bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b3f0 tx=0x7f09fc00b8b0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 shutdown_connections 2024-09-11T22:13:35.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f09f80a2c10 0x7f09f80a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a0c105f40 0x7f0a0c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a0c101450 0x7f0a0c1a4bf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 --2- 172.21.15.92:0/4445258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c100b30 0x7f0a0c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 >> 172.21.15.92:0/4445258 conn(0x7f0a0c0fc7a0 msgr2=0x7f0a0c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 shutdown_connections 2024-09-11T22:13:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:35.680+0000 7f0a13d34640 1 -- 172.21.15.92:0/4445258 wait complete. 2024-09-11T22:13:35.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:35 smithi092 ceph-mon[31173]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:35.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:35 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4445258' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:13:36.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:35 smithi110 ceph-mon[34830]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:36.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:35 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4445258' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:13:36.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:35 smithi153 ceph-mon[35045]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:36.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:35 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4445258' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:13:36.317 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T22:13:36.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:13:36.566 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:36.954 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:36 smithi092 ceph-mon[31173]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:37.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 -- 172.21.15.92:0/58883680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718106a80 msgr2=0x7f5718106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:37.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 --2- 172.21.15.92:0/58883680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718106a80 0x7f5718106e60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5714009930 tx=0x7f571402a420 comp rx=0 tx=0).stop 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 -- 172.21.15.92:0/58883680 shutdown_connections 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 --2- 172.21.15.92:0/58883680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5718101420 0x7f571810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 --2- 172.21.15.92:0/58883680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5718100a80 0x7f5718100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 --2- 172.21.15.92:0/58883680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718106a80 0x7f5718106e60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 -- 172.21.15.92:0/58883680 >> 172.21.15.92:0/58883680 conn(0x7f57180fc7a0 msgr2=0x7f57180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 -- 172.21.15.92:0/58883680 shutdown_connections 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.072+0000 7f571f960640 1 -- 172.21.15.92:0/58883680 wait complete. 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 Processor -- start 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 -- start start 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718100a80 0x7f5718199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5718101420 0x7f5718199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:37.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5718106a80 0x7f57181a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5718110d30 con 0x7f5718100a80 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5718110bb0 con 0x7f5718106a80 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571f960640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5718110eb0 con 0x7f5718101420 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718100a80 0x7f5718199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718100a80 0x7f5718199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51474/0 (socket says 172.21.15.92:51474) 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 -- 172.21.15.92:0/2283893698 learned_addr learned my addr 172.21.15.92:0/2283893698 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571cefb640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5718101420 0x7f5718199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571defd640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5718106a80 0x7f57181a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 -- 172.21.15.92:0/2283893698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5718101420 msgr2=0x7f5718199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5718101420 0x7f5718199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 -- 172.21.15.92:0/2283893698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5718106a80 msgr2=0x7f57181a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5718106a80 0x7f57181a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571d6fc640 1 -- 172.21.15.92:0/2283893698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57181a47c0 con 0x7f5718100a80 2024-09-11T22:13:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571cefb640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5718101420 0x7f5718199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.073+0000 7f571defd640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5718106a80 0x7f57181a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.074+0000 7f571d6fc640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718100a80 0x7f5718199200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5714004770 tx=0x7f57140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.074+0000 7f570e7fc640 1 -- 172.21.15.92:0/2283893698 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571403f070 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.074+0000 7f570e7fc640 1 -- 172.21.15.92:0/2283893698 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f571403b460 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.074+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57181a4990 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.074+0000 7f570e7fc640 1 -- 172.21.15.92:0/2283893698 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714035070 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.074+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181a4e70 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.075+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5718100ee0 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.076+0000 7f570e7fc640 1 -- 172.21.15.92:0/2283893698 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f571403b610 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.077+0000 7f570e7fc640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f57040a2c90 0x7f57040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.077+0000 7f571cefb640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f57040a2c90 0x7f57040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.077+0000 7f570e7fc640 1 -- 172.21.15.92:0/2283893698 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f57140ef920 con 0x7f5718100a80 2024-09-11T22:13:37.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.077+0000 7f571cefb640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f57040a2c90 0x7f57040a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57080058c0 tx=0x7f5708005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:37.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.081+0000 7f570e7fc640 1 -- 172.21.15.92:0/2283893698 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57140bc3c0 con 0x7f5718100a80 2024-09-11T22:13:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:36 smithi110 ceph-mon[34830]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:37.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.282+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f571802d7c0 con 0x7f5718100a80 2024-09-11T22:13:37.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.282+0000 7f570e7fc640 1 -- 172.21.15.92:0/2283893698 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11980 (secure 0 0 0) 0x7f57140bbcc0 con 0x7f5718100a80 2024-09-11T22:13:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:37.284 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"btime":"2024-09-11T22:12:40:147657+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:40.147644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:13:37.284 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f57040a2c90 msgr2=0x7f57040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f57040a2c90 0x7f57040a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57080058c0 tx=0x7f5708005850 comp rx=0 tx=0).stop 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718100a80 msgr2=0x7f5718199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718100a80 0x7f5718199200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5714004770 tx=0x7f57140042f0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 shutdown_connections 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f57040a2c90 0x7f57040a5150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5718106a80 0x7f57181a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5718101420 0x7f5718199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 --2- 172.21.15.92:0/2283893698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718100a80 0x7f5718199200 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:37.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 >> 172.21.15.92:0/2283893698 conn(0x7f57180fc7a0 msgr2=0x7f57180fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:37.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 shutdown_connections 2024-09-11T22:13:37.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:37.285+0000 7f571f960640 1 -- 172.21.15.92:0/2283893698 wait complete. 2024-09-11T22:13:37.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:36 smithi153 ceph-mon[35045]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:37.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:13:37.967 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:37 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2283893698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:38.189 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:37 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2283893698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:38.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:37 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2283893698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 -- 172.21.15.92:0/2224729595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 msgr2=0x7f3b6410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/2224729595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b6410ea20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009900 tx=0x7f3b4c02a410 comp rx=0 tx=0).stop 2024-09-11T22:13:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 -- 172.21.15.92:0/2224729595 shutdown_connections 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/2224729595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b6410ea20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/2224729595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b64100100 0x7f3b64100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/2224729595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b64106b80 0x7f3b64106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 -- 172.21.15.92:0/2224729595 >> 172.21.15.92:0/2224729595 conn(0x7f3b64077230 msgr2=0x7f3b640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 -- 172.21.15.92:0/2224729595 shutdown_connections 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 -- 172.21.15.92:0/2224729595 wait complete. 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 Processor -- start 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.664+0000 7f3b6aae8640 1 -- start start 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b6aae8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b64100100 0x7f3b641982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b6aae8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b64198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b6aae8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b64106b80 0x7f3b6419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b6aae8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64111f00 con 0x7f3b64100aa0 2024-09-11T22:13:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b6aae8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64111d80 con 0x7f3b64106b80 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b6aae8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64112080 con 0x7f3b64100100 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b64198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b64198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51484/0 (socket says 172.21.15.92:51484) 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 -- 172.21.15.92:0/4017399809 learned_addr learned my addr 172.21.15.92:0/4017399809 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b69085640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b64106b80 0x7f3b6419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b68884640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b64100100 0x7f3b641982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 -- 172.21.15.92:0/4017399809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b64100100 msgr2=0x7f3b641982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b64100100 0x7f3b641982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 -- 172.21.15.92:0/4017399809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b64106b80 msgr2=0x7f3b6419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b64106b80 0x7f3b6419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 -- 172.21.15.92:0/4017399809 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6419d290 con 0x7f3b64100aa0 2024-09-11T22:13:38.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b63fff640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b64198820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400cce0 tx=0x7f3b5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b61ffb640 1 -- 172.21.15.92:0/4017399809 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5400f650 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6419d580 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b61ffb640 1 -- 172.21.15.92:0/4017399809 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b5400fca0 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.665+0000 7f3b61ffb640 1 -- 172.21.15.92:0/4017399809 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54018b90 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.666+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b641a4e60 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.667+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b64100560 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.668+0000 7f3b61ffb640 1 -- 172.21.15.92:0/4017399809 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3b5400f7f0 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.668+0000 7f3b61ffb640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b500a2c10 0x7f3b500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.668+0000 7f3b61ffb640 1 -- 172.21.15.92:0/4017399809 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3b540cfb20 con 0x7f3b64100aa0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.668+0000 7f3b68884640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b500a2c10 0x7f3b500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.669+0000 7f3b68884640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b500a2c10 0x7f3b500a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c005fd0 tx=0x7f3b5c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.672+0000 7f3b61ffb640 1 -- 172.21.15.92:0/4017399809 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b54014030 con 0x7f3b64100aa0 2024-09-11T22:13:38.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.879+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b6410a010 con 0x7f3b64100aa0 2024-09-11T22:13:38.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.880+0000 7f3b61ffb640 1 -- 172.21.15.92:0/4017399809 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11980 (secure 0 0 0) 0x7f3b5409c560 con 0x7f3b64100aa0 2024-09-11T22:13:38.881 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:38.881 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"btime":"2024-09-11T22:12:40:147657+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:40.147644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:13:38.882 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-09-11T22:13:38.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b500a2c10 msgr2=0x7f3b500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:38.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b500a2c10 0x7f3b500a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c005fd0 tx=0x7f3b5c005850 comp rx=0 tx=0).stop 2024-09-11T22:13:38.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 msgr2=0x7f3b64198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:38.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b64198820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400cce0 tx=0x7f3b5400a620 comp rx=0 tx=0).stop 2024-09-11T22:13:38.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 shutdown_connections 2024-09-11T22:13:38.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f3b500a2c10 0x7f3b500a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b64106b80 0x7f3b6419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b64100aa0 0x7f3b64198820 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 --2- 172.21.15.92:0/4017399809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b64100100 0x7f3b641982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:38.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 >> 172.21.15.92:0/4017399809 conn(0x7f3b64077230 msgr2=0x7f3b640fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:38.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 shutdown_connections 2024-09-11T22:13:38.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:38.882+0000 7f3b6aae8640 1 -- 172.21.15.92:0/4017399809 wait complete. 2024-09-11T22:13:39.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:38 smithi092 ceph-mon[31173]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:38 smithi110 ceph-mon[34830]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:39.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:38 smithi153 ceph-mon[35045]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:39.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:13:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:40.148 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:39 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4017399809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:40.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:39 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4017399809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:40.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:39 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4017399809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:40.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 -- 172.21.15.92:0/2583345535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 msgr2=0x7fbca01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:40.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2583345535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca01021c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009900 tx=0x7fbc9002a410 comp rx=0 tx=0).stop 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 -- 172.21.15.92:0/2583345535 shutdown_connections 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2583345535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbca0106f10 0x7fbca010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2583345535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca0102790 0x7fbca01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2583345535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca01021c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 -- 172.21.15.92:0/2583345535 >> 172.21.15.92:0/2583345535 conn(0x7fbca00fd8c0 msgr2=0x7fbca00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 -- 172.21.15.92:0/2583345535 shutdown_connections 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.357+0000 7fbca5fce640 1 -- 172.21.15.92:0/2583345535 wait complete. 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 Processor -- start 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 -- start start 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbca0102790 0x7fbca0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca0106f10 0x7fbca0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:40.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0111e20 con 0x7fbca0101de0 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0111ca0 con 0x7fbca0106f10 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbca5fce640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0111fa0 con 0x7fbca0102790 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51502/0 (socket says 172.21.15.92:51502) 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbc9f7fe640 1 -- 172.21.15.92:0/2202326592 learned_addr learned my addr 172.21.15.92:0/2202326592 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbc9effd640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbca0102790 0x7fbca0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.358+0000 7fbc9ffff640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca0106f10 0x7fbca0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9f7fe640 1 -- 172.21.15.92:0/2202326592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbca0102790 msgr2=0x7fbca0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9f7fe640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbca0102790 0x7fbca0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9f7fe640 1 -- 172.21.15.92:0/2202326592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca0106f10 msgr2=0x7fbca0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9f7fe640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca0106f10 0x7fbca0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9f7fe640 1 -- 172.21.15.92:0/2202326592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca01a6820 con 0x7fbca0101de0 2024-09-11T22:13:40.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9ffff640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca0106f10 0x7fbca0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9f7fe640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca00744f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbc90005940 tx=0x7fbc90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9cff9640 1 -- 172.21.15.92:0/2202326592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9003f030 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca01a6ab0 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9cff9640 1 -- 172.21.15.92:0/2202326592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc90003aa0 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbc9cff9640 1 -- 172.21.15.92:0/2202326592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90035070 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.359+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a6fe0 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.360+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca01021c0 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.361+0000 7fbc9cff9640 1 -- 172.21.15.92:0/2202326592 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbc90042030 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.362+0000 7fbc9cff9640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbc880a2c10 0x7fbc880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.362+0000 7fbc9effd640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbc880a2c10 0x7fbc880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.362+0000 7fbc9cff9640 1 -- 172.21.15.92:0/2202326592 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbc900efce0 con 0x7fbca0101de0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.362+0000 7fbc9effd640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbc880a2c10 0x7fbc880a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbca0075f80 tx=0x7fbc9400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:40.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.366+0000 7fbc9cff9640 1 -- 172.21.15.92:0/2202326592 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc900bc780 con 0x7fbca0101de0 2024-09-11T22:13:40.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.534+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbca010a0f0 con 0x7fbca0101de0 2024-09-11T22:13:40.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.535+0000 7fbc9cff9640 1 -- 172.21.15.92:0/2202326592 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24737 (secure 0 0 0) 0x7fbc900bc080 con 0x7fbca0101de0 2024-09-11T22:13:40.537 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:40.537 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:13:16.772187+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:13:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbc880a2c10 msgr2=0x7fbc880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbc880a2c10 0x7fbc880a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbca0075f80 tx=0x7fbc9400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 msgr2=0x7fbca00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca00744f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbc90005940 tx=0x7fbc90004270 comp rx=0 tx=0).stop 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 shutdown_connections 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fbc880a2c10 0x7fbc880a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca0106f10 0x7fbca0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbca0102790 0x7fbca0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 --2- 172.21.15.92:0/2202326592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca0101de0 0x7fbca00744f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 >> 172.21.15.92:0/2202326592 conn(0x7fbca00fd8c0 msgr2=0x7fbca00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 shutdown_connections 2024-09-11T22:13:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:40.537+0000 7fbca5fce640 1 -- 172.21.15.92:0/2202326592 wait complete. 2024-09-11T22:13:41.176 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:40 smithi092 ceph-mon[31173]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:13:41.177 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:40 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2202326592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:41.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:13:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:40 smithi110 ceph-mon[34830]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:13:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:40 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2202326592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:41.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:40 smithi153 ceph-mon[35045]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:13:41.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:40 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2202326592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:13:41.449 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 -- 172.21.15.92:0/2579305914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 msgr2=0x7f645c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 --2- 172.21.15.92:0/2579305914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c10da40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6454009ca0 tx=0x7f645402a950 comp rx=0 tx=0).stop 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 -- 172.21.15.92:0/2579305914 shutdown_connections 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 --2- 172.21.15.92:0/2579305914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c10da40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 --2- 172.21.15.92:0/2579305914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f645c102570 0x7f645c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 --2- 172.21.15.92:0/2579305914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f645c100e20 0x7f645c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 -- 172.21.15.92:0/2579305914 >> 172.21.15.92:0/2579305914 conn(0x7f645c0fc7d0 msgr2=0x7f645c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 -- 172.21.15.92:0/2579305914 shutdown_connections 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.932+0000 7f6462791640 1 -- 172.21.15.92:0/2579305914 wait complete. 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 Processor -- start 2024-09-11T22:13:41.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 -- start start 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f645c100e20 0x7f645c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f645c102570 0x7f645c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f645c110e30 con 0x7f645c102ef0 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f645c110cb0 con 0x7f645c100e20 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.933+0000 7f6462791640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f645c110fb0 con 0x7f645c102570 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c19bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51520/0 (socket says 172.21.15.92:51520) 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 -- 172.21.15.92:0/3503558249 learned_addr learned my addr 172.21.15.92:0/3503558249 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f645bfff640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f645c100e20 0x7f645c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f645b7fe640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f645c102570 0x7f645c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 -- 172.21.15.92:0/3503558249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f645c102570 msgr2=0x7f645c197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f645c102570 0x7f645c197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 -- 172.21.15.92:0/3503558249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f645c100e20 msgr2=0x7f645c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f645c100e20 0x7f645c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 -- 172.21.15.92:0/3503558249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c19c300 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6460d2e640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c19bc20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f645402ae70 tx=0x7f645402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f64597fa640 1 -- 172.21.15.92:0/3503558249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454034040 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f64597fa640 1 -- 172.21.15.92:0/3503558249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64540308f0 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c19c590 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f64597fa640 1 -- 172.21.15.92:0/3503558249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454004780 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.934+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c19ca10 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.935+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c101200 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.937+0000 7f64597fa640 1 -- 172.21.15.92:0/3503558249 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6454037020 con 0x7f645c102ef0 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.938+0000 7f64597fa640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f64440a2c10 0x7f64440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:41.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.938+0000 7f645bfff640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f64440a2c10 0x7f64440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.938+0000 7f64597fa640 1 -- 172.21.15.92:0/3503558249 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f64540f2020 con 0x7f645c102ef0 2024-09-11T22:13:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.938+0000 7f645bfff640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f64440a2c10 0x7f64440a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f644c005fd0 tx=0x7f644c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:41.941+0000 7f64597fa640 1 -- 172.21.15.92:0/3503558249 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6454040070 con 0x7f645c102ef0 2024-09-11T22:13:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.139+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f645c108c50 con 0x7f645c102ef0 2024-09-11T22:13:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.140+0000 7f64597fa640 1 -- 172.21.15.92:0/3503558249 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11980 (secure 0 0 0) 0x7f64540be470 con 0x7f645c102ef0 2024-09-11T22:13:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:42.142 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"btime":"2024-09-11T22:12:40:147657+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:40.147644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:13:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f64440a2c10 msgr2=0x7f64440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f64440a2c10 0x7f64440a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f644c005fd0 tx=0x7f644c00b390 comp rx=0 tx=0).stop 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 msgr2=0x7f645c19bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c19bc20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f645402ae70 tx=0x7f645402f5d0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 shutdown_connections 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f64440a2c10 0x7f64440a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f645c102ef0 0x7f645c19bc20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f645c102570 0x7f645c197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 --2- 172.21.15.92:0/3503558249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f645c100e20 0x7f645c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.142+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 >> 172.21.15.92:0/3503558249 conn(0x7f645c0fc7d0 msgr2=0x7f645c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.143+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 shutdown_connections 2024-09-11T22:13:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:42.143+0000 7f6462791640 1 -- 172.21.15.92:0/3503558249 wait complete. 2024-09-11T22:13:42.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:12:03.420066+0000', 'modified': '2024-09-11T22:12:40.147644+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24455, 'mds_1': 24451, 'mds_2': 24433, 'mds_3': 24428, 'mds_4': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14607': {'gid': 14607, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6839/1790524059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1790524059}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1790524059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14619': {'gid': 14619, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6841/2477024139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6840', 'nonce': 2477024139}, {'type': 'v1', 'addr': '172.21.15.92:6841', 'nonce': 2477024139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6833/797341789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 797341789}, {'type': 'v1', 'addr': '172.21.15.110:6833', 'nonce': 797341789}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6833/490387556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 490387556}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 490387556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/1455978865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 1455978865}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 1455978865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.153:6835/1638990868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 1638990868}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 1638990868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.110:6837/3571280995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 3571280995}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 3571280995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.153:6837/1434479445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 1434479445}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 1434479445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.153:6839/229095273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 229095273}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 229095273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.110:6839/538708966', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 538708966}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 538708966}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24455, 'qdb_cluster': [24455, 24451, 24433, 24428, 24427]} 2024-09-11T22:13:42.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:13:42.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-11T22:13:43.168 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:42 smithi110 ceph-mon[34830]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:13:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:42 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3503558249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:42 smithi092 ceph-mon[31173]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:13:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:42 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3503558249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:43.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:42 smithi153 ceph-mon[35045]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:13:43.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:42 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3503558249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:13:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 -- 172.21.15.92:0/1968185824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308101de0 msgr2=0x7f33081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 --2- 172.21.15.92:0/1968185824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308101de0 0x7f33081021c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009900 tx=0x7f32fc02a410 comp rx=0 tx=0).stop 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 -- 172.21.15.92:0/1968185824 shutdown_connections 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 --2- 172.21.15.92:0/1968185824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3308106f10 0x7f330810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 --2- 172.21.15.92:0/1968185824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3308102790 0x7f33081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 --2- 172.21.15.92:0/1968185824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308101de0 0x7f33081021c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 -- 172.21.15.92:0/1968185824 >> 172.21.15.92:0/1968185824 conn(0x7f33080fd8c0 msgr2=0x7f33080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 -- 172.21.15.92:0/1968185824 shutdown_connections 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.619+0000 7f330edae640 1 -- 172.21.15.92:0/1968185824 wait complete. 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 Processor -- start 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 -- start start 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3308101de0 0x7f33080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308102790 0x7f3308074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3308106f10 0x7f3308074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3308111e20 con 0x7f3308102790 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3308111ca0 con 0x7f3308106f10 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330edae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3308111fa0 con 0x7f3308101de0 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308102790 0x7f3308074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308102790 0x7f3308074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51538/0 (socket says 172.21.15.92:51538) 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 -- 172.21.15.92:0/2522789379 learned_addr learned my addr 172.21.15.92:0/2522789379 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330cb4a640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3308101de0 0x7f33080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f330d34b640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3308106f10 0x7f3308074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 -- 172.21.15.92:0/2522789379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3308101de0 msgr2=0x7f33080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3308101de0 0x7f33080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 -- 172.21.15.92:0/2522789379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3308106f10 msgr2=0x7f3308074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3308106f10 0x7f3308074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.620+0000 7f3307fff640 1 -- 172.21.15.92:0/2522789379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081a6820 con 0x7f3308102790 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.621+0000 7f3307fff640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308102790 0x7f3308074a30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c890 tx=0x7f32f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.621+0000 7f3305ffb640 1 -- 172.21.15.92:0/2522789379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4004220 con 0x7f3308102790 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.621+0000 7f3305ffb640 1 -- 172.21.15.92:0/2522789379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f40043c0 con 0x7f3308102790 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.621+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33081a6b10 con 0x7f3308102790 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.621+0000 7f3305ffb640 1 -- 172.21.15.92:0/2522789379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4004a00 con 0x7f3308102790 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.621+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081a70a0 con 0x7f3308102790 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3305ffb640 1 -- 172.21.15.92:0/2522789379 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f32f4004ba0 con 0x7f3308102790 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 Processor -- start 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 -- start start 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc952fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc953a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc953cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558cc9496090 con 0x558cc953abb0 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558cc943da70 con 0x558cc952f9d0 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3311187740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558cc9492050 con 0x558cc95302f0 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.624+0000 7f3307fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc953cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc953cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51550/0 (socket says 172.21.15.92:51550) 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 -- 172.21.15.92:0/918686132 learned_addr learned my addr 172.21.15.92:0/918686132 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f330d34b640 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc952fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3305ffb640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32f80a2c90 0x7f32f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f330cb4a640 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc953a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f330d34b640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32f80a2c90 0x7f32f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3305ffb640 1 -- 172.21.15.92:0/2522789379 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32f40cf170 con 0x7f3308102790 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 -- 172.21.15.92:0/918686132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 msgr2=0x558cc953a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc953a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 -- 172.21.15.92:0/918686132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 msgr2=0x558cc952fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc952fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 -- 172.21.15.92:0/918686132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc944a130 con 0x558cc953abb0 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f330cb4a640 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc953a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f330d34b640 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc952fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f330d34b640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32f80a2c90 0x7f32f80a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3300012b80 tx=0x7f3300004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f3307fff640 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc953cfa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f32f40a3c50 tx=0x7f32f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b052849731dac4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f32f2ffd640 1 -- 172.21.15.92:0/918686132 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f401d030 con 0x558cc953abb0 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f32f2ffd640 1 -- 172.21.15.92:0/918686132 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f409bc10 con 0x558cc953abb0 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.625+0000 7f32f2ffd640 1 -- 172.21.15.92:0/918686132 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40050b0 con 0x558cc953abb0 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 -- 172.21.15.92:0/918686132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 msgr2=0x558cc953cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc953cfa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f32f40a3c50 tx=0x7f32f4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 -- 172.21.15.92:0/918686132 shutdown_connections 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc953cfa0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc953a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 --2- 172.21.15.92:0/918686132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc952fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 -- 172.21.15.92:0/918686132 >> 172.21.15.92:0/918686132 conn(0x558cc952bf60 msgr2=0x558cc952c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 -- 172.21.15.92:0/918686132 shutdown_connections 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 -- 172.21.15.92:0/918686132 wait complete. 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.626+0000 7f3311187740 1 -- start start 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 1 Processor -- start 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc953a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc95302f0 0x558cc9537780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc953abb0 0x558cc9537cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558cc953d780 con 0x558cc95302f0 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558cc9450f20 con 0x558cc952f9d0 2024-09-11T22:13:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558cc953d900 con 0x558cc953abb0 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3311187740 10 client.0 _ll_register_callbacks cb 0x558cc941a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc95302f0 0x558cc9537780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc95302f0 0x558cc9537780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51562/0 (socket says 172.21.15.92:51562) 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 -- 172.21.15.92:0/450790334 learned_addr learned my addr 172.21.15.92:0/450790334 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f330cb4a640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc953a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f330d34b640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc953abb0 0x558cc9537cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 -- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc953abb0 msgr2=0x558cc9537cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc953abb0 0x558cc9537cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 -- 172.21.15.92:0/450790334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 msgr2=0x558cc953a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc953a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f3307fff640 1 -- 172.21.15.92:0/450790334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc92072b0 con 0x558cc95302f0 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f330cb4a640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc953a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.627+0000 7f330d34b640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc953abb0 0x558cc9537cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3307fff640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc95302f0 0x558cc9537780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32f40971c0 tx=0x7f32f409aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80187c298e9f6f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f32e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f32e7fff640 1 -- 172.21.15.92:0/450790334 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f401d040 con 0x558cc95302f0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f32e7fff640 1 -- 172.21.15.92:0/450790334 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32f40050b0 con 0x558cc95302f0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f409bc10 con 0x558cc95302f0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f32e7fff640 1 -- 172.21.15.92:0/450790334 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32f409f8c0 con 0x558cc95302f0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558cc953d780 con 0x558cc95302f0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f32e7fff640 1 -- 172.21.15.92:0/450790334 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f409bc10 con 0x558cc95302f0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f32e7fff640 1 -- 172.21.15.92:0/450790334 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f32f40b4220 con 0x558cc95302f0 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 10 client.15075 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T22:13:43.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 20 client.15075 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc954c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 4 client.15075 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.628+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 --> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cc94a3600 con 0x558cc954b220 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.629+0000 7f330cb4a640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc954c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.629+0000 7f32e7fff640 10 client.15075 ms_handle_connect on v2:172.21.15.153:6834/1638990868 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.629+0000 7f330cb4a640 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc954c690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62c1fd6e50c43957 server_cookie=e0d3ddf3e7d57ba3 in_seq=0 out_seq=0 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f32e7fff640 1 -- 172.21.15.92:0/450790334 <== mds.4 v2:172.21.15.153:6834/1638990868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cc94a3600 con 0x558cc954b220 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f32e7fff640 10 client.15075 handle_command_reply: tid=0 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 client.15075 shutdown 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc95302f0 msgr2=0x558cc9537780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc95302f0 0x558cc9537780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32f40971c0 tx=0x7f32f409aef0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 shutdown_connections 2024-09-11T22:13:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc954c690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc953abb0 0x558cc9537cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f32e7fff640 0 client.15075 ms_handle_reset on v2:172.21.15.153:6834/1638990868 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc95302f0 0x558cc9537780 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 --2- 172.21.15.92:0/450790334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc952f9d0 0x558cc953a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 >> 172.21.15.92:0/450790334 conn(0x558cc952bf60 msgr2=0x558cc9531c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 shutdown_connections 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 1 -- 172.21.15.92:0/450790334 wait complete. 2024-09-11T22:13:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.630+0000 7f3311187740 20 client.15075 trim_cache size 0 max 16384 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 Processor -- start 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 -- start start 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc952f9d0 0x558cc954c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc95349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc953abb0 0x558cc9534f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558cc954a9d0 con 0x558cc952f9d0 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558cc954b0a0 con 0x558cc953abb0 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3311187740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558cc954b770 con 0x558cc95302f0 2024-09-11T22:13:43.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f330cb4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc952f9d0 0x558cc954c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f330cb4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc952f9d0 0x558cc954c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51580/0 (socket says 172.21.15.92:51580) 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f330cb4a640 1 -- 172.21.15.92:0/219867807 learned_addr learned my addr 172.21.15.92:0/219867807 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.648+0000 7f3307fff640 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc95349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330cb4a640 1 -- 172.21.15.92:0/219867807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 msgr2=0x558cc95349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330d34b640 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc953abb0 0x558cc9534f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330cb4a640 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc95349e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330cb4a640 1 -- 172.21.15.92:0/219867807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc953abb0 msgr2=0x558cc9534f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330cb4a640 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc953abb0 0x558cc9534f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330cb4a640 1 -- 172.21.15.92:0/219867807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc9538540 con 0x558cc952f9d0 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330d34b640 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc953abb0 0x558cc9534f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f330cb4a640 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc952f9d0 0x558cc954c690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f32fc02a930 tx=0x7f32fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda28631dbaa2658 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f32f27fc640 1 -- 172.21.15.92:0/219867807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc048070 con 0x558cc952f9d0 2024-09-11T22:13:43.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f32f27fc640 1 -- 172.21.15.92:0/219867807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32fc0035f0 con 0x558cc952f9d0 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.649+0000 7f32f27fc640 1 -- 172.21.15.92:0/219867807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc035040 con 0x558cc952f9d0 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 -- 172.21.15.92:0/219867807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc952f9d0 msgr2=0x558cc954c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc952f9d0 0x558cc954c690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f32fc02a930 tx=0x7f32fc005e80 comp rx=0 tx=0).stop 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 -- 172.21.15.92:0/219867807 shutdown_connections 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc953abb0 0x558cc9534f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc95302f0 0x558cc95349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 --2- 172.21.15.92:0/219867807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc952f9d0 0x558cc954c690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 -- 172.21.15.92:0/219867807 >> 172.21.15.92:0/219867807 conn(0x558cc952bf60 msgr2=0x558cc954a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 -- 172.21.15.92:0/219867807 shutdown_connections 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 -- 172.21.15.92:0/219867807 wait complete. 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 -- start start 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 Processor -- start 2024-09-11T22:13:43.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc952f9d0 0x558cc9554ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc95302f0 0x558cc9555020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.650+0000 7f3311187740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc954dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f3311187740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558cc954c510 con 0x558cc953abb0 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f3311187740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558cc954be40 con 0x558cc95302f0 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f3311187740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558cc954cbe0 con 0x558cc952f9d0 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f3311187740 10 client.0 _ll_register_callbacks cb 0x7f33000084d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc954dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc954dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51582/0 (socket says 172.21.15.92:51582) 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 -- 172.21.15.92:0/747544401 learned_addr learned my addr 172.21.15.92:0/747544401 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f3307fff640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc95302f0 0x558cc9555020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330cb4a640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc952f9d0 0x558cc9554ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 -- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc952f9d0 msgr2=0x558cc9554ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc952f9d0 0x558cc9554ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 -- 172.21.15.92:0/747544401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc95302f0 msgr2=0x558cc9555020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc95302f0 0x558cc9555020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 -- 172.21.15.92:0/747544401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc954e6d0 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330cb4a640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc952f9d0 0x558cc9554ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f330d34b640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc954dea0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3300007a30 tx=0x7f3300004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e0ce8a66ad4f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f32e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f32e7fff640 1 -- 172.21.15.92:0/747544401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300004d20 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f32e7fff640 1 -- 172.21.15.92:0/747544401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3300003970 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.651+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32fc0035f0 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f32e7fff640 1 -- 172.21.15.92:0/747544401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3300032050 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558cc9553090 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f32e7fff640 1 -- 172.21.15.92:0/747544401 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32fc0035f0 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f32e7fff640 1 -- 172.21.15.92:0/747544401 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f3300005140 con 0x558cc953abb0 2024-09-11T22:13:43.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 10 client.15081 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 20 client.15081 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc95b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 4 client.15081 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 --> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] -- command(tid 0: {"prefix": "status"}) -- 0x558cc949db10 con 0x558cc954b220 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.652+0000 7f330cb4a640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc95b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f330cb4a640 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc95b8d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=562c72f4a7adf8d0 server_cookie=a39af909183e4c9c in_seq=0 out_seq=0 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f32e7fff640 10 client.15081 ms_handle_connect on v2:172.21.15.153:6834/1638990868 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f32e7fff640 1 -- 172.21.15.92:0/747544401 <== mds.4 v2:172.21.15.153:6834/1638990868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558cc949db10 con 0x558cc954b220 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f32e7fff640 10 client.15081 handle_command_reply: tid=0 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 client.15081 shutdown 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 msgr2=0x558cc954dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc954dea0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3300007a30 tx=0x7f3300004a40 comp rx=0 tx=0).stop 2024-09-11T22:13:43.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 shutdown_connections 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x558cc954b220 0x558cc95b8d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cc953abb0 0x558cc954dea0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cc95302f0 0x558cc9555020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 --2- 172.21.15.92:0/747544401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cc952f9d0 0x558cc9554ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f32e7fff640 0 client.15081 ms_handle_reset on v2:172.21.15.153:6834/1638990868 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 >> 172.21.15.92:0/747544401 conn(0x558cc952bf60 msgr2=0x558cc95325c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 shutdown_connections 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 1 -- 172.21.15.92:0/747544401 wait complete. 2024-09-11T22:13:43.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.653+0000 7f3311187740 20 client.15081 trim_cache size 0 max 16384 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24427, 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 64.509928735000003, 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 126.83820698300001 2024-09-11T22:13:43.662 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:13:43.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32f80a2c90 msgr2=0x7f32f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32f80a2c90 0x7f32f80a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3300012b80 tx=0x7f3300004ab0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308102790 msgr2=0x7f3308074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308102790 0x7f3308074a30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c890 tx=0x7f32f400cd50 comp rx=0 tx=0).stop 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 shutdown_connections 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f32f80a2c90 0x7f32f80a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3308106f10 0x7f3308074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308102790 0x7f3308074a30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 --2- 172.21.15.92:0/2522789379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3308101de0 0x7f33080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 >> 172.21.15.92:0/2522789379 conn(0x7f33080fd8c0 msgr2=0x7f33080fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 shutdown_connections 2024-09-11T22:13:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:43.654+0000 7f330edae640 1 -- 172.21.15.92:0/2522789379 wait complete. 2024-09-11T22:13:44.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:13:44.657 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:45.042 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:44 smithi092 ceph-mon[31173]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:45.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 -- 172.21.15.92:0/1170025704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 msgr2=0x7f62c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 --2- 172.21.15.92:0/1170025704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c4101200 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009900 tx=0x7f62c002a480 comp rx=0 tx=0).stop 2024-09-11T22:13:45.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 -- 172.21.15.92:0/1170025704 shutdown_connections 2024-09-11T22:13:45.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 --2- 172.21.15.92:0/1170025704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c4102ef0 0x7f62c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 --2- 172.21.15.92:0/1170025704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c4102570 0x7f62c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 --2- 172.21.15.92:0/1170025704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c4101200 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 -- 172.21.15.92:0/1170025704 >> 172.21.15.92:0/1170025704 conn(0x7f62c40fc7d0 msgr2=0x7f62c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 -- 172.21.15.92:0/1170025704 shutdown_connections 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.123+0000 7f62cc184640 1 -- 172.21.15.92:0/1170025704 wait complete. 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 Processor -- start 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 -- start start 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c4102570 0x7f62c41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c4102ef0 0x7f62c41a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f62c4110ea0 con 0x7f62c4100e20 2024-09-11T22:13:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62c4110d20 con 0x7f62c4102570 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62cc184640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62c4111020 con 0x7f62c4102ef0 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33018/0 (socket says 172.21.15.92:33018) 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 -- 172.21.15.92:0/626604073 learned_addr learned my addr 172.21.15.92:0/626604073 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62ca721640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c4102ef0 0x7f62c41a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c971f640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c4102570 0x7f62c41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 -- 172.21.15.92:0/626604073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c4102ef0 msgr2=0x7f62c41a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c4102ef0 0x7f62c41a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 -- 172.21.15.92:0/626604073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c4102570 msgr2=0x7f62c41997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c4102570 0x7f62c41997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62c9f20640 1 -- 172.21.15.92:0/626604073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c41a4870 con 0x7f62c4100e20 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.124+0000 7f62ca721640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c4102ef0 0x7f62c41a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.125+0000 7f62c9f20640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c41992b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.125+0000 7f62baffd640 1 -- 172.21.15.92:0/626604073 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c003f070 con 0x7f62c4100e20 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.125+0000 7f62baffd640 1 -- 172.21.15.92:0/626604073 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c003bd20 con 0x7f62c4100e20 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.125+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c41a4a40 con 0x7f62c4100e20 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.125+0000 7f62baffd640 1 -- 172.21.15.92:0/626604073 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035030 con 0x7f62c4100e20 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.125+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c41a4e90 con 0x7f62c4100e20 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.127+0000 7f62baffd640 1 -- 172.21.15.92:0/626604073 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f62c0042070 con 0x7f62c4100e20 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.127+0000 7f62ce55d740 1 Processor -- start 2024-09-11T22:13:45.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ce55d740 1 -- start start 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62baffd640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f62b00a2c10 0x7f62b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e3261facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fb1f0 0x55e326209a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e32620c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ce55d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e326161090 con 0x55e3261fb1f0 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ce55d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e326108a70 con 0x55e3261fa8d0 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ce55d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e32615d050 con 0x55e326209fc0 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62baffd640 1 -- 172.21.15.92:0/626604073 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f62c00eff10 con 0x7f62c4100e20 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ca721640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f62b00a2c10 0x7f62b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62c971f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fb1f0 0x55e326209a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62ca721640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e32620c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62c971f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fb1f0 0x55e326209a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33024/0 (socket says 172.21.15.92:33024) 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62c971f640 1 -- 172.21.15.92:0/916330329 learned_addr learned my addr 172.21.15.92:0/916330329 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.128+0000 7f62c9f20640 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e3261facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62c971f640 1 -- 172.21.15.92:0/916330329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 msgr2=0x55e32620c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62c971f640 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e32620c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62c971f640 1 -- 172.21.15.92:0/916330329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 msgr2=0x55e3261facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62c971f640 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e3261facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62c971f640 1 -- 172.21.15.92:0/916330329 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e326115130 con 0x55e3261fb1f0 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62ca721640 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e32620c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62c9f20640 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e3261facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62ca721640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f62b00a2c10 0x7f62b00a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f62bc012750 tx=0x7f62bc0079d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62c971f640 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fb1f0 0x55e326209a80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b7f0 tx=0x7f62b400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea2e315354a05cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62affff640 1 -- 172.21.15.92:0/916330329 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b400d950 con 0x55e3261fb1f0 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62affff640 1 -- 172.21.15.92:0/916330329 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b4018070 con 0x55e3261fb1f0 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.129+0000 7f62affff640 1 -- 172.21.15.92:0/916330329 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4013e30 con 0x55e3261fb1f0 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 -- 172.21.15.92:0/916330329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fb1f0 msgr2=0x55e326209a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fb1f0 0x55e326209a80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b7f0 tx=0x7f62b400bcb0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 -- 172.21.15.92:0/916330329 shutdown_connections 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e32620c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fb1f0 0x55e326209a80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 --2- 172.21.15.92:0/916330329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e3261facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 -- 172.21.15.92:0/916330329 >> 172.21.15.92:0/916330329 conn(0x55e3261f6f60 msgr2=0x55e3261f7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 -- 172.21.15.92:0/916330329 shutdown_connections 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 -- 172.21.15.92:0/916330329 wait complete. 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 -- start start 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.130+0000 7f62ce55d740 1 Processor -- start 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e326209840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fb1f0 0x55e326206b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e3262070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ce55d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e32620cbf0 con 0x55e3261fa8d0 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ce55d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e32611bf20 con 0x55e3261fb1f0 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ce55d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e32620cd70 con 0x55e326209fc0 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ce55d740 10 client.0 _ll_register_callbacks cb 0x55e3260e5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:45.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e326209840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e326209840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33036/0 (socket says 172.21.15.92:33036) 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 -- 172.21.15.92:0/3705527018 learned_addr learned my addr 172.21.15.92:0/3705527018 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c971f640 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fb1f0 0x55e326206b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ca721640 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e3262070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 -- 172.21.15.92:0/3705527018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 msgr2=0x55e3262070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e3262070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 -- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fb1f0 msgr2=0x55e326206b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fb1f0 0x55e326206b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 -- 172.21.15.92:0/3705527018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e325ed22b0 con 0x55e3261fa8d0 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62c9f20640 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e326209840 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f62c00bc240 tx=0x7f62c00bbdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954f22362763ea4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.131+0000 7f62ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ad7fa640 1 -- 172.21.15.92:0/3705527018 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0049040 con 0x55e3261fa8d0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ad7fa640 1 -- 172.21.15.92:0/3705527018 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f62c00b8920 con 0x55e3261fa8d0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b4018070 con 0x55e3261fa8d0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ad7fa640 1 -- 172.21.15.92:0/3705527018 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62c00da050 con 0x55e3261fa8d0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e326218570 con 0x55e3261fa8d0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ad7fa640 1 -- 172.21.15.92:0/3705527018 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b4018070 con 0x55e3261fa8d0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ad7fa640 1 -- 172.21.15.92:0/3705527018 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f62c002e260 con 0x55e3261fa8d0 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 10 client.15093 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 20 client.15093 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e32621c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 4 client.15093 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.132+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 --> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e32616e600 con 0x55e32621abf0 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.133+0000 7f62c971f640 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e32621c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.133+0000 7f62ad7fa640 10 client.15093 ms_handle_connect on v2:172.21.15.110:6836/3571280995 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.133+0000 7f62c971f640 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e32621c060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb13582df9badaf9 server_cookie=f2239607b412ff2d in_seq=0 out_seq=0 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ad7fa640 1 -- 172.21.15.92:0/3705527018 <== mds.3 v2:172.21.15.110:6836/3571280995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e32616e600 con 0x55e32621abf0 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ad7fa640 10 client.15093 handle_command_reply: tid=0 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 client.15093 shutdown 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 msgr2=0x55e326209840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e326209840 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f62c00bc240 tx=0x7f62c00bbdf0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 shutdown_connections 2024-09-11T22:13:45.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e32621c060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e326209fc0 0x55e3262070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ad7fa640 0 client.15093 ms_handle_reset on v2:172.21.15.110:6836/3571280995 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fb1f0 0x55e326206b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 --2- 172.21.15.92:0/3705527018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e326209840 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 >> 172.21.15.92:0/3705527018 conn(0x55e3261f6f60 msgr2=0x55e3261fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 shutdown_connections 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 1 -- 172.21.15.92:0/3705527018 wait complete. 2024-09-11T22:13:45.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.134+0000 7f62ce55d740 20 client.15093 trim_cache size 0 max 16384 2024-09-11T22:13:45.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 Processor -- start 2024-09-11T22:13:45.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 -- start start 2024-09-11T22:13:45.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e32621c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e3261fba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e326209fc0 0x55e326206450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e32621a3a0 con 0x55e326209fc0 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e32621aa70 con 0x55e3261fa8d0 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.152+0000 7f62ce55d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e32621b140 con 0x55e3261fb1f0 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e326209fc0 0x55e326206450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e326209fc0 0x55e326206450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33050/0 (socket says 172.21.15.92:33050) 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 -- 172.21.15.92:0/2204675752 learned_addr learned my addr 172.21.15.92:0/2204675752 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62c971f640 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e3261fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62c9f20640 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e32621c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 -- 172.21.15.92:0/2204675752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 msgr2=0x55e3261fba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e3261fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 -- 172.21.15.92:0/2204675752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 msgr2=0x55e32621c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e32621c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 -- 172.21.15.92:0/2204675752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e326207950 con 0x55e326209fc0 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62c9f20640 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e32621c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62ca721640 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e326209fc0 0x55e326206450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f62bc003c00 tx=0x7f62bc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309dd52bd1b5d7e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62af7fe640 1 -- 172.21.15.92:0/2204675752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc004040 con 0x55e326209fc0 2024-09-11T22:13:45.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62af7fe640 1 -- 172.21.15.92:0/2204675752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62bc004690 con 0x55e326209fc0 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.153+0000 7f62af7fe640 1 -- 172.21.15.92:0/2204675752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc020c50 con 0x55e326209fc0 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 -- 172.21.15.92:0/2204675752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e326209fc0 msgr2=0x55e326206450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e326209fc0 0x55e326206450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f62bc003c00 tx=0x7f62bc01b000 comp rx=0 tx=0).stop 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 -- 172.21.15.92:0/2204675752 shutdown_connections 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e326209fc0 0x55e326206450 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e3261fba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 --2- 172.21.15.92:0/2204675752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3261fa8d0 0x55e32621c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 -- 172.21.15.92:0/2204675752 >> 172.21.15.92:0/2204675752 conn(0x55e3261f6f60 msgr2=0x55e3261f7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 -- 172.21.15.92:0/2204675752 shutdown_connections 2024-09-11T22:13:45.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 -- 172.21.15.92:0/2204675752 wait complete. 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.154+0000 7f62ce55d740 1 -- start start 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 1 Processor -- start 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e32621d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e32621d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326209fc0 0x55e32621dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e32621bee0 con 0x55e3261fa8d0 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e32621b810 con 0x55e326209fc0 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e32621c5b0 con 0x55e3261fb1f0 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ce55d740 10 client.0 _ll_register_callbacks cb 0x7f62c0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:45.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e32621d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e32621d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33062/0 (socket says 172.21.15.92:33062) 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 -- 172.21.15.92:0/977028478 learned_addr learned my addr 172.21.15.92:0/977028478 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ca721640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326209fc0 0x55e32621dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c971f640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e32621d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 -- 172.21.15.92:0/977028478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 msgr2=0x55e32621d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e32621d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 -- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326209fc0 msgr2=0x55e32621dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326209fc0 0x55e32621dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62c9f20640 1 -- 172.21.15.92:0/977028478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e326283540 con 0x55e3261fa8d0 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.155+0000 7f62ca721640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326209fc0 0x55e32621dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62c9f20640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e32621d2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f62c0036040 tx=0x7f62c00c07a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102025251c4e463 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ad7fa640 1 -- 172.21.15.92:0/977028478 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0049040 con 0x55e3261fa8d0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ad7fa640 1 -- 172.21.15.92:0/977028478 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f62c00b8920 con 0x55e3261fa8d0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62bc004690 con 0x55e3261fa8d0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ad7fa640 1 -- 172.21.15.92:0/977028478 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62c00bbc10 con 0x55e3261fa8d0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e326284790 con 0x55e3261fa8d0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ad7fa640 1 -- 172.21.15.92:0/977028478 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62bc004690 con 0x55e3261fa8d0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ad7fa640 1 -- 172.21.15.92:0/977028478 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f62c00c4a90 con 0x55e3261fa8d0 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 10 client.15099 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 20 client.15099 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e326287e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 4 client.15099 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.156+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 --> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] -- command(tid 0: {"prefix": "status"}) -- 0x55e326168b10 con 0x55e32621abf0 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62c971f640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e326287e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ad7fa640 10 client.15099 ms_handle_connect on v2:172.21.15.110:6836/3571280995 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62c971f640 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e326287e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d2fa87ff301ed3e server_cookie=833c1366ad4d6149 in_seq=0 out_seq=0 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ad7fa640 1 -- 172.21.15.92:0/977028478 <== mds.3 v2:172.21.15.110:6836/3571280995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e326168b10 con 0x55e32621abf0 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ad7fa640 10 client.15099 handle_command_reply: tid=0 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 client.15099 shutdown 2024-09-11T22:13:45.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 msgr2=0x55e32621d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e32621d2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f62c0036040 tx=0x7f62c00c07a0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 shutdown_connections 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326209fc0 0x55e32621dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55e32621abf0 0x55e326287e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e3261fb1f0 0x55e32621d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ce55d740 1 --2- 172.21.15.92:0/977028478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3261fa8d0 0x55e32621d2c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.157+0000 7f62ad7fa640 0 client.15099 ms_handle_reset on v2:172.21.15.110:6836/3571280995 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 >> 172.21.15.92:0/977028478 conn(0x55e3261f6f60 msgr2=0x55e326205390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 shutdown_connections 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62ce55d740 1 -- 172.21.15.92:0/977028478 wait complete. 2024-09-11T22:13:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62ce55d740 20 client.15099 trim_cache size 0 max 16384 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24428, 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:13:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 67.024014871999995, 2024-09-11T22:13:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:13:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:13:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:13:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 123.135949321 2024-09-11T22:13:45.177 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f62b00a2c10 msgr2=0x7f62b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f62b00a2c10 0x7f62b00a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f62bc012750 tx=0x7f62bc0079d0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 msgr2=0x7f62c41992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c41992b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c0003d50 comp rx=0 tx=0).stop 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 shutdown_connections 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f62b00a2c10 0x7f62b00a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c4102ef0 0x7f62c41a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c4102570 0x7f62c41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 --2- 172.21.15.92:0/626604073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4100e20 0x7f62c41992b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 >> 172.21.15.92:0/626604073 conn(0x7f62c40fc7d0 msgr2=0x7f62c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 shutdown_connections 2024-09-11T22:13:45.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:45.158+0000 7f62cc184640 1 -- 172.21.15.92:0/626604073 wait complete. 2024-09-11T22:13:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:44 smithi110 ceph-mon[34830]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:45.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:44 smithi153 ceph-mon[35045]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:45.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:13:46.180 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:46.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.690+0000 7f24f721c640 1 -- 172.21.15.92:0/1439955329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0106f10 msgr2=0x7f24f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.690+0000 7f24f721c640 1 --2- 172.21.15.92:0/1439955329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0106f10 0x7f24f010ea90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009ca0 tx=0x7f24e802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- 172.21.15.92:0/1439955329 shutdown_connections 2024-09-11T22:13:46.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 --2- 172.21.15.92:0/1439955329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0106f10 0x7f24f010ea90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 --2- 172.21.15.92:0/1439955329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f0102790 0x7f24f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 --2- 172.21.15.92:0/1439955329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f0101de0 0x7f24f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- 172.21.15.92:0/1439955329 >> 172.21.15.92:0/1439955329 conn(0x7f24f00fd8c0 msgr2=0x7f24f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- 172.21.15.92:0/1439955329 shutdown_connections 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- 172.21.15.92:0/1439955329 wait complete. 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 Processor -- start 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- start start 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f0101de0 0x7f24f0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0102790 0x7f24f01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f0106f10 0x7f24f019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0111e20 con 0x7f24f0102790 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0111ca0 con 0x7f24f0106f10 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24f721c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0111fa0 con 0x7f24f0101de0 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.691+0000 7f24effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0102790 0x7f24f01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0102790 0x7f24f01988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33074/0 (socket says 172.21.15.92:33074) 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 -- 172.21.15.92:0/1727198401 learned_addr learned my addr 172.21.15.92:0/1727198401 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f0101de0 0x7f24f0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24f57b9640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f0106f10 0x7f24f019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 -- 172.21.15.92:0/1727198401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f0101de0 msgr2=0x7f24f0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f0101de0 0x7f24f0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 -- 172.21.15.92:0/1727198401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f0106f10 msgr2=0x7f24f019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f0106f10 0x7f24f019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 -- 172.21.15.92:0/1727198401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f019d310 con 0x7f24f0102790 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24effff640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0102790 0x7f24f01988a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00d950 tx=0x7f24dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24edffb640 1 -- 172.21.15.92:0/1727198401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc017070 con 0x7f24f0102790 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f019d600 con 0x7f24f0102790 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24edffb640 1 -- 172.21.15.92:0/1727198401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24dc009c30 con 0x7f24f0102790 2024-09-11T22:13:46.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24edffb640 1 -- 172.21.15.92:0/1727198401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00f620 con 0x7f24f0102790 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.692+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01a4ee0 con 0x7f24f0102790 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.694+0000 7f24edffb640 1 -- 172.21.15.92:0/1727198401 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f24dc0109a0 con 0x7f24f0102790 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 Processor -- start 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 -- start start 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24edffb640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f24e00a2c10 0x7f24e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae8fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fe2f0 0x5627ae908670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae90afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5627ae864090 con 0x5627ae8fe2f0 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5627ae80ba70 con 0x5627ae908bb0 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f95f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627ae860050 con 0x5627ae8fd9d0 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f57b9640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f24e00a2c10 0x7f24e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24edffb640 1 -- 172.21.15.92:0/1727198401 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24dc0cf110 con 0x7f24f0102790 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fe2f0 0x5627ae908670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fe2f0 0x5627ae908670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33078/0 (socket says 172.21.15.92:33078) 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24effff640 1 -- 172.21.15.92:0/646789186 learned_addr learned my addr 172.21.15.92:0/646789186 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.695+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae8fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24f57b9640 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae90afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24effff640 1 -- 172.21.15.92:0/646789186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 msgr2=0x5627ae8fddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24effff640 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae8fddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24effff640 1 -- 172.21.15.92:0/646789186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 msgr2=0x5627ae90afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24effff640 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae90afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24effff640 1 -- 172.21.15.92:0/646789186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ae818130 con 0x5627ae8fe2f0 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24f57b9640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f24e00a2c10 0x7f24e00a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24e8006010 tx=0x7f24e8031b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24effff640 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fe2f0 0x5627ae908670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0ccce0 tx=0x7f24dc09b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34571c09b487f1e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24daffd640 1 -- 172.21.15.92:0/646789186 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc020070 con 0x5627ae8fe2f0 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24daffd640 1 -- 172.21.15.92:0/646789186 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24dc09bbb0 con 0x5627ae8fe2f0 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.696+0000 7f24daffd640 1 -- 172.21.15.92:0/646789186 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc0cc9b0 con 0x5627ae8fe2f0 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 -- 172.21.15.92:0/646789186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fe2f0 msgr2=0x5627ae908670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fe2f0 0x5627ae908670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0ccce0 tx=0x7f24dc09b120 comp rx=0 tx=0).stop 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 -- 172.21.15.92:0/646789186 shutdown_connections 2024-09-11T22:13:46.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae90afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fe2f0 0x5627ae908670 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 --2- 172.21.15.92:0/646789186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae8fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 -- 172.21.15.92:0/646789186 >> 172.21.15.92:0/646789186 conn(0x5627ae8f9f60 msgr2=0x5627ae8fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 -- 172.21.15.92:0/646789186 shutdown_connections 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 -- 172.21.15.92:0/646789186 wait complete. 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 -- start start 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.697+0000 7f24f95f5740 1 Processor -- start 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae9083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae905730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae905c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5627ae90fb90 con 0x5627ae8fd9d0 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5627ae81ef20 con 0x5627ae908bb0 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627ae90fd10 con 0x5627ae8fe2f0 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 10 client.0 _ll_register_callbacks cb 0x5627ae7e8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae9083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae9083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33084/0 (socket says 172.21.15.92:33084) 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3471798706 learned_addr learned my addr 172.21.15.92:0/3471798706 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24effff640 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae905730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f57b9640 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae905c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 msgr2=0x5627ae905730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae905730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3471798706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 msgr2=0x5627ae905c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae905c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3471798706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ae5d52b0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae9083e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f24e400ba80 tx=0x7f24e400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793da60f97cdb396 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24cffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24cffff640 1 -- 172.21.15.92:0/3471798706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4011440 con 0x5627ae8fd9d0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24cffff640 1 -- 172.21.15.92:0/3471798706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24e40115e0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.698+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24dc09bbb0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24cffff640 1 -- 172.21.15.92:0/3471798706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f24e4011820 con 0x5627ae8fd9d0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627ae91b4a0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24cffff640 1 -- 172.21.15.92:0/3471798706 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24dc09bbb0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24cffff640 1 -- 172.21.15.92:0/3471798706 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f24e401ba30 con 0x5627ae8fd9d0 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 20 client.15111 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae91db30 0x5627ae91efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 4 client.15111 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 --> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627ae871600 con 0x5627ae91db30 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.699+0000 7f24effff640 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae91db30 0x5627ae91efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.700+0000 7f24effff640 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae91db30 0x5627ae91efa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f817998bca68d716 server_cookie=26596d70d8dadea7 in_seq=0 out_seq=0 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.700+0000 7f24cffff640 10 client.15111 ms_handle_connect on v2:172.21.15.153:6836/1434479445 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.700+0000 7f24cffff640 1 -- 172.21.15.92:0/3471798706 <== mds.2 v2:172.21.15.153:6836/1434479445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627ae871600 con 0x5627ae91db30 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.700+0000 7f24cffff640 10 client.15111 handle_command_reply: tid=0 2024-09-11T22:13:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.700+0000 7f24f95f5740 1 client.15111 shutdown 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 msgr2=0x5627ae9083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae9083e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f24e400ba80 tx=0x7f24e400bf40 comp rx=0 tx=0).stop 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 shutdown_connections 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae91db30 0x5627ae91efa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae905c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae905730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24cffff640 0 client.15111 ms_handle_reset on v2:172.21.15.153:6836/1434479445 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3471798706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae9083e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 >> 172.21.15.92:0/3471798706 conn(0x5627ae8f9f60 msgr2=0x5627ae8ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 shutdown_connections 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 1 -- 172.21.15.92:0/3471798706 wait complete. 2024-09-11T22:13:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.701+0000 7f24f95f5740 20 client.15111 trim_cache size 0 max 16384 2024-09-11T22:13:46.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.718+0000 7f24f95f5740 1 Processor -- start 2024-09-11T22:13:46.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.718+0000 7f24f95f5740 1 -- start start 2024-09-11T22:13:46.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae91efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae8fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae907590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f95f5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5627ae91d2e0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f95f5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5627ae91d9b0 con 0x5627ae908bb0 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f95f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627ae91e080 con 0x5627ae8fe2f0 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae91efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae91efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33098/0 (socket says 172.21.15.92:33098) 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3688073876 learned_addr learned my addr 172.21.15.92:0/3688073876 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f57b9640 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae907590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24effff640 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae8fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3688073876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 msgr2=0x5627ae8fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae8fa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3688073876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 msgr2=0x5627ae907590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae907590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 -- 172.21.15.92:0/3688073876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ae908750 con 0x5627ae8fd9d0 2024-09-11T22:13:46.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f57b9640 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae907590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae91efa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24e4003f70 tx=0x7f24e4003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e269c52b927bfb74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24da7fc640 1 -- 172.21.15.92:0/3688073876 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4011440 con 0x5627ae8fd9d0 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24da7fc640 1 -- 172.21.15.92:0/3688073876 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e40115e0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.719+0000 7f24da7fc640 1 -- 172.21.15.92:0/3688073876 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4010e60 con 0x5627ae8fd9d0 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 -- 172.21.15.92:0/3688073876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 msgr2=0x5627ae91efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae91efa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24e4003f70 tx=0x7f24e4003fa0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 -- 172.21.15.92:0/3688073876 shutdown_connections 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae908bb0 0x5627ae907590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fe2f0 0x5627ae8fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3688073876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae8fd9d0 0x5627ae91efa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 -- 172.21.15.92:0/3688073876 >> 172.21.15.92:0/3688073876 conn(0x5627ae8f9f60 msgr2=0x5627ae906e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 -- 172.21.15.92:0/3688073876 shutdown_connections 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.720+0000 7f24f95f5740 1 -- 172.21.15.92:0/3688073876 wait complete. 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 -- start start 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 Processor -- start 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae9277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae8fe2f0 0x5627ae922b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae908bb0 0x5627ae923060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5627ae91ee20 con 0x5627ae908bb0 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5627ae91e750 con 0x5627ae8fe2f0 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627ae91f4f0 con 0x5627ae8fd9d0 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f95f5740 10 client.0 _ll_register_callbacks cb 0x7f24e4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f57b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae908bb0 0x5627ae923060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24effff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae8fe2f0 0x5627ae922b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f57b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae908bb0 0x5627ae923060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33100/0 (socket says 172.21.15.92:33100) 2024-09-11T22:13:46.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f57b9640 1 -- 172.21.15.92:0/3909350305 learned_addr learned my addr 172.21.15.92:0/3909350305 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.721+0000 7f24f4fb8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae9277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24effff640 1 -- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 msgr2=0x5627ae9277f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24effff640 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae9277f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24effff640 1 -- 172.21.15.92:0/3909350305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae908bb0 msgr2=0x5627ae923060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24effff640 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae908bb0 0x5627ae923060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24effff640 1 -- 172.21.15.92:0/3909350305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ae9236a0 con 0x5627ae8fe2f0 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae9277f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24effff640 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae8fe2f0 0x5627ae922b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24dc09b670 tx=0x7f24dc005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a463c49d927623d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24cffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24cffff640 1 -- 172.21.15.92:0/3909350305 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc020040 con 0x5627ae8fe2f0 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e40115e0 con 0x5627ae8fe2f0 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24cffff640 1 -- 172.21.15.92:0/3909350305 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24dc097b20 con 0x5627ae8fe2f0 2024-09-11T22:13:46.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24cffff640 1 -- 172.21.15.92:0/3909350305 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f24dc09bbb0 con 0x5627ae8fe2f0 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24f95f5740 10 client.24710 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627ae988000 con 0x5627ae8fe2f0 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.722+0000 7f24cffff640 1 -- 172.21.15.92:0/3909350305 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e40115e0 con 0x5627ae8fe2f0 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24cffff640 1 -- 172.21.15.92:0/3909350305 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f24dc0864a0 con 0x5627ae8fe2f0 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24f95f5740 10 client.24710 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24f95f5740 10 client.24710 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24f95f5740 20 client.24710 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae98c720 0x5627ae98b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24f95f5740 4 client.24710 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 --> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] -- command(tid 0: {"prefix": "status"}) -- 0x5627ae86bb10 con 0x5627ae98c720 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.723+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae98c720 0x5627ae98b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f4fb8640 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae98c720 0x5627ae98b430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61470b5b795a8554 server_cookie=93cd9d79710714c6 in_seq=0 out_seq=0 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24cffff640 10 client.24710 ms_handle_connect on v2:172.21.15.153:6836/1434479445 2024-09-11T22:13:46.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24cffff640 1 -- 172.21.15.92:0/3909350305 <== mds.2 v2:172.21.15.153:6836/1434479445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5627ae86bb10 con 0x5627ae98c720 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24cffff640 10 client.24710 handle_command_reply: tid=0 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 client.24710 shutdown 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae8fe2f0 msgr2=0x5627ae922b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae8fe2f0 0x5627ae922b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24dc09b670 tx=0x7f24dc005880 comp rx=0 tx=0).stop 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 shutdown_connections 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x5627ae98c720 0x5627ae98b430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627ae908bb0 0x5627ae923060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627ae8fe2f0 0x5627ae922b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24cffff640 0 client.24710 ms_handle_reset on v2:172.21.15.153:6836/1434479445 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 --2- 172.21.15.92:0/3909350305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627ae8fd9d0 0x5627ae9277f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 >> 172.21.15.92:0/3909350305 conn(0x5627ae8f9f60 msgr2=0x5627ae8f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 shutdown_connections 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 1 -- 172.21.15.92:0/3909350305 wait complete. 2024-09-11T22:13:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.724+0000 7f24f95f5740 20 client.24710 trim_cache size 0 max 16384 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24433, 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 69.600861363999996, 2024-09-11T22:13:46.735 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:13:46.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:13:46.736 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:13:46.736 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 127.338869227 2024-09-11T22:13:46.736 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:13:46.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f24e00a2c10 msgr2=0x7f24e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f24e00a2c10 0x7f24e00a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24e8006010 tx=0x7f24e8031b20 comp rx=0 tx=0).stop 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0102790 msgr2=0x7f24f01988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0102790 0x7f24f01988a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00d950 tx=0x7f24dc00de10 comp rx=0 tx=0).stop 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 shutdown_connections 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f24e00a2c10 0x7f24e00a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f0106f10 0x7f24f019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24f0102790 0x7f24f01988a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 --2- 172.21.15.92:0/1727198401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f0101de0 0x7f24f0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 >> 172.21.15.92:0/1727198401 conn(0x7f24f00fd8c0 msgr2=0x7f24f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 shutdown_connections 2024-09-11T22:13:46.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:46.725+0000 7f24f721c640 1 -- 172.21.15.92:0/1727198401 wait complete. 2024-09-11T22:13:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:46 smithi110 ceph-mon[34830]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:46 smithi092 ceph-mon[31173]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:46 smithi153 ceph-mon[35045]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:47.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:13:47.698 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:48.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 -- 172.21.15.92:0/723606689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 msgr2=0x7efee410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/723606689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee410d7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efedc009cd0 tx=0x7efedc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 -- 172.21.15.92:0/723606689 shutdown_connections 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/723606689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee410d7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/723606689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efee41008d0 0x7efee4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/723606689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee41068d0 0x7efee4106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 -- 172.21.15.92:0/723606689 >> 172.21.15.92:0/723606689 conn(0x7efee40fc610 msgr2=0x7efee40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 -- 172.21.15.92:0/723606689 shutdown_connections 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 -- 172.21.15.92:0/723606689 wait complete. 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 Processor -- start 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.166+0000 7efeeb3a2640 1 -- start start 2024-09-11T22:13:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeeb3a2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee41008d0 0x7efee4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeeb3a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee4197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeeb3a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efee41068d0 0x7efee419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeeb3a2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110b80 con 0x7efee4101270 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeeb3a2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110a00 con 0x7efee41068d0 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeeb3a2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110d00 con 0x7efee41008d0 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee4197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee4197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33120/0 (socket says 172.21.15.92:33120) 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 -- 172.21.15.92:0/1587026804 learned_addr learned my addr 172.21.15.92:0/1587026804 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeeaba1640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efee41068d0 0x7efee419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeea3a0640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee41008d0 0x7efee4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 -- 172.21.15.92:0/1587026804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee41008d0 msgr2=0x7efee4197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee41008d0 0x7efee4197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 -- 172.21.15.92:0/1587026804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efee41068d0 msgr2=0x7efee419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efee41068d0 0x7efee419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 -- 172.21.15.92:0/1587026804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee419bff0 con 0x7efee4101270 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efeea3a0640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee41008d0 0x7efee4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efee9b9f640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee4197580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efee0007b50 tx=0x7efee000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.167+0000 7efed77fe640 1 -- 172.21.15.92:0/1587026804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0007c40 con 0x7efee4101270 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.168+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee419c2e0 con 0x7efee4101270 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.168+0000 7efed77fe640 1 -- 172.21.15.92:0/1587026804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee0007de0 con 0x7efee4101270 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.168+0000 7efed77fe640 1 -- 172.21.15.92:0/1587026804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee000ad40 con 0x7efee4101270 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.168+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee4069f70 con 0x7efee4101270 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.170+0000 7efed77fe640 1 -- 172.21.15.92:0/1587026804 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efee000eb30 con 0x7efee4101270 2024-09-11T22:13:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.170+0000 7efeee9dd740 1 Processor -- start 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.170+0000 7efeee9dd740 1 -- start start 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efed77fe640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efed00a2c10 0x7efed00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e140870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e140db0 0x55e65e1431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeee9dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e65e09c090 con 0x55e65e135bd0 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeee9dd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e65e043a70 con 0x55e65e1364f0 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeee9dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e65e098050 con 0x55e65e140db0 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeeaba1640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efed00a2c10 0x7efed00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efed77fe640 1 -- 172.21.15.92:0/1587026804 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efee00d0bb0 con 0x7efee4101270 2024-09-11T22:13:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e135fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33122/0 (socket says 172.21.15.92:33122) 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 -- 172.21.15.92:0/2985373694 learned_addr learned my addr 172.21.15.92:0/2985373694 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efee9b9f640 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 -- 172.21.15.92:0/2985373694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e140db0 msgr2=0x55e65e1431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e140db0 0x55e65e1431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 -- 172.21.15.92:0/2985373694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 msgr2=0x55e65e140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 -- 172.21.15.92:0/2985373694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65e050130 con 0x55e65e135bd0 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeeaba1640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efed00a2c10 0x7efed00a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efedc006010 tx=0x7efedc005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efeea3a0640 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e135fb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efed8009b50 tx=0x7efed800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f58a59b6c714a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efed4ff9640 1 -- 172.21.15.92:0/2985373694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8016070 con 0x55e65e135bd0 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efed4ff9640 1 -- 172.21.15.92:0/2985373694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed8009e40 con 0x55e65e135bd0 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.171+0000 7efed4ff9640 1 -- 172.21.15.92:0/2985373694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8010730 con 0x55e65e135bd0 2024-09-11T22:13:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 -- 172.21.15.92:0/2985373694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 msgr2=0x55e65e135fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e135fb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efed8009b50 tx=0x7efed800e6c0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 -- 172.21.15.92:0/2985373694 shutdown_connections 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e140db0 0x55e65e1431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 --2- 172.21.15.92:0/2985373694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e135fb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 -- 172.21.15.92:0/2985373694 >> 172.21.15.92:0/2985373694 conn(0x55e65e132160 msgr2=0x55e65e132540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 -- 172.21.15.92:0/2985373694 shutdown_connections 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.172+0000 7efeee9dd740 1 -- 172.21.15.92:0/2985373694 wait complete. 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 -- start start 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 Processor -- start 2024-09-11T22:13:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e140680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e13d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e13de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e65e143980 con 0x55e65e140db0 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e65e056f20 con 0x55e65e1364f0 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e65e143b00 con 0x55e65e135bd0 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeee9dd740 10 client.0 _ll_register_callbacks cb 0x55e65e020030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e13de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e13de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33134/0 (socket says 172.21.15.92:33134) 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 -- 172.21.15.92:0/893859772 learned_addr learned my addr 172.21.15.92:0/893859772 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeea3a0640 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efee9b9f640 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e13d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 -- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 msgr2=0x55e65e140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e140680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 -- 172.21.15.92:0/893859772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 msgr2=0x55e65e13d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e13d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.173+0000 7efeeaba1640 1 -- 172.21.15.92:0/893859772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65de0d2b0 con 0x55e65e140db0 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeeaba1640 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e13de70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efedc009cd0 tx=0x7efedc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe47d838448f1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efecdffb640 1 -- 172.21.15.92:0/893859772 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc04d070 con 0x55e65e140db0 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e65e0a9600 con 0x55e65e140db0 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efecdffb640 1 -- 172.21.15.92:0/893859772 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efedc0305a0 con 0x55e65e140db0 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efecdffb640 1 -- 172.21.15.92:0/893859772 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efedc059050 con 0x55e65e140db0 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e65e14f320 con 0x55e65e140db0 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efecdffb640 1 -- 172.21.15.92:0/893859772 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e65e0a9600 con 0x55e65e140db0 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efecdffb640 1 -- 172.21.15.92:0/893859772 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7efedc030070 con 0x55e65e140db0 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 20 client.15129 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e151950 0x55e65e152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 4 client.15129 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.174+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 --> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e65e0a3b10 con 0x55e65e151950 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.175+0000 7efeea3a0640 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e151950 0x55e65e152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.175+0000 7efeea3a0640 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e151950 0x55e65e152dc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee3c40f8b2adb7d6 server_cookie=fd1b0fbd02e41964 in_seq=0 out_seq=0 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.175+0000 7efecdffb640 10 client.15129 ms_handle_connect on v2:172.21.15.153:6838/229095273 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efecdffb640 1 -- 172.21.15.92:0/893859772 <== mds.1 v2:172.21.15.153:6838/229095273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e65e0a3b10 con 0x55e65e151950 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efecdffb640 10 client.15129 handle_command_reply: tid=0 2024-09-11T22:13:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 client.15129 shutdown 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 msgr2=0x55e65e13de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e13de70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efedc009cd0 tx=0x7efedc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 shutdown_connections 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e13de70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e13d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e151950 0x55e65e152dc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 --2- 172.21.15.92:0/893859772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e140680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 >> 172.21.15.92:0/893859772 conn(0x55e65e132160 msgr2=0x55e65e137e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efecdffb640 0 client.15129 ms_handle_reset on v2:172.21.15.153:6838/229095273 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 shutdown_connections 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 1 -- 172.21.15.92:0/893859772 wait complete. 2024-09-11T22:13:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.176+0000 7efeee9dd740 20 client.15129 trim_cache size 0 max 16384 2024-09-11T22:13:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 Processor -- start 2024-09-11T22:13:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 -- start start 2024-09-11T22:13:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e151950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e1364f0 0x55e65e132540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e140db0 0x55e65e13abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e65e151100 con 0x55e65e135bd0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e65e1517d0 con 0x55e65e140db0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeee9dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e65e151ea0 con 0x55e65e1364f0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33146/0 (socket says 172.21.15.92:33146) 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 -- 172.21.15.92:0/1042833111 learned_addr learned my addr 172.21.15.92:0/1042833111 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efee9b9f640 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e1364f0 0x55e65e132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 -- 172.21.15.92:0/1042833111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e1364f0 msgr2=0x55e65e132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e1364f0 0x55e65e132540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 -- 172.21.15.92:0/1042833111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e140db0 msgr2=0x55e65e13abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e140db0 0x55e65e13abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efeea3a0640 1 -- 172.21.15.92:0/1042833111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65e1402a0 con 0x55e65e135bd0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.194+0000 7efee9b9f640 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e1364f0 0x55e65e132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeea3a0640 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e151950 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efed8009af0 tx=0x7efed80103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d702300e453cbcba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efecffff640 1 -- 172.21.15.92:0/1042833111 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8016040 con 0x55e65e135bd0 2024-09-11T22:13:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efecffff640 1 -- 172.21.15.92:0/1042833111 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed8005520 con 0x55e65e135bd0 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efecffff640 1 -- 172.21.15.92:0/1042833111 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed80116d0 con 0x55e65e135bd0 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 -- 172.21.15.92:0/1042833111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 msgr2=0x55e65e151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e151950 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efed8009af0 tx=0x7efed80103d0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 -- 172.21.15.92:0/1042833111 shutdown_connections 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e140db0 0x55e65e13abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e1364f0 0x55e65e132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 --2- 172.21.15.92:0/1042833111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e135bd0 0x55e65e151950 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 -- 172.21.15.92:0/1042833111 >> 172.21.15.92:0/1042833111 conn(0x55e65e132160 msgr2=0x55e65e138fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 -- 172.21.15.92:0/1042833111 shutdown_connections 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.195+0000 7efeee9dd740 1 -- 172.21.15.92:0/1042833111 wait complete. 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 -- start start 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 Processor -- start 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e154090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e65e152c40 con 0x55e65e140db0 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e65e152570 con 0x55e65e1364f0 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e65e153310 con 0x55e65e135bd0 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeee9dd740 10 client.0 _ll_register_callbacks cb 0x7efedc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeeaba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeeaba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33152/0 (socket says 172.21.15.92:33152) 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeeaba1640 1 -- 172.21.15.92:0/117633232 learned_addr learned my addr 172.21.15.92:0/117633232 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efeea3a0640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.196+0000 7efee9b9f640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeeaba1640 1 -- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 msgr2=0x55e65e154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeeaba1640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeeaba1640 1 -- 172.21.15.92:0/117633232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 msgr2=0x55e65e1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeeaba1640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e1545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeeaba1640 1 -- 172.21.15.92:0/117633232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65e1ba2f0 con 0x55e65e140db0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efee9b9f640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeeaba1640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e154b10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efedc030360 tx=0x7efedc02b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5212b792cd5aa447 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efecdffb640 1 -- 172.21.15.92:0/117633232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc062060 con 0x55e65e140db0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efecdffb640 1 -- 172.21.15.92:0/117633232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efedc03d8e0 con 0x55e65e140db0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed8005520 con 0x55e65e140db0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efecdffb640 1 -- 172.21.15.92:0/117633232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efedc069050 con 0x55e65e140db0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeee9dd740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.197+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e65e1bb560 con 0x55e65e140db0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efecdffb640 1 -- 172.21.15.92:0/117633232 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed8005520 con 0x55e65e140db0 2024-09-11T22:13:48.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efecdffb640 1 -- 172.21.15.92:0/117633232 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7efedc030040 con 0x55e65e140db0 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeee9dd740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeee9dd740 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeee9dd740 20 client.15135 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeee9dd740 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e1c00f0 0x55e65e1bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeee9dd740 4 client.15135 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 --> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] -- command(tid 0: {"prefix": "status"}) -- 0x55e65e0200e0 con 0x55e65e1c00f0 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeea3a0640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e1c00f0 0x55e65e1bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efeea3a0640 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e1c00f0 0x55e65e1bee00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=482f2857ff480214 server_cookie=add00826b250ce1b in_seq=0 out_seq=0 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.198+0000 7efecdffb640 10 client.15135 ms_handle_connect on v2:172.21.15.153:6838/229095273 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efecdffb640 1 -- 172.21.15.92:0/117633232 <== mds.1 v2:172.21.15.153:6838/229095273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e65e0200e0 con 0x55e65e1c00f0 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efecdffb640 10 client.15135 handle_command_reply: tid=0 2024-09-11T22:13:48.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 client.15135 shutdown 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 msgr2=0x55e65e154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e154b10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efedc030360 tx=0x7efedc02b020 comp rx=0 tx=0).stop 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 shutdown_connections 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e65e140db0 0x55e65e154b10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e65e1364f0 0x55e65e1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55e65e1c00f0 0x55e65e1bee00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efecdffb640 0 client.15135 ms_handle_reset on v2:172.21.15.153:6838/229095273 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 --2- 172.21.15.92:0/117633232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e65e135bd0 0x55e65e154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 >> 172.21.15.92:0/117633232 conn(0x55e65e132160 msgr2=0x55e65e137620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 shutdown_connections 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 1 -- 172.21.15.92:0/117633232 wait complete. 2024-09-11T22:13:48.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeee9dd740 20 client.15135 trim_cache size 0 max 16384 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24451, 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 72.08120486, 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:13:48.208 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:13:48.209 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:13:48.209 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 123.67497174099999 2024-09-11T22:13:48.209 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:13:48.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efed00a2c10 msgr2=0x7efed00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efed00a2c10 0x7efed00a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efedc006010 tx=0x7efedc005ef0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 msgr2=0x7efee4197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.199+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee4197580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efee0007b50 tx=0x7efee000e420 comp rx=0 tx=0).stop 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 shutdown_connections 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efed00a2c10 0x7efed00a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efee41068d0 0x7efee419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee4101270 0x7efee4197580 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 --2- 172.21.15.92:0/1587026804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee41008d0 0x7efee4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 >> 172.21.15.92:0/1587026804 conn(0x7efee40fc610 msgr2=0x7efee40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 shutdown_connections 2024-09-11T22:13:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:48.200+0000 7efeeb3a2640 1 -- 172.21.15.92:0/1587026804 wait complete. 2024-09-11T22:13:48.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:13:49.165 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:48 smithi110 ceph-mon[34830]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:48 smithi092 ceph-mon[31173]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:49.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:48 smithi153 ceph-mon[35045]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:49.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 -- 172.21.15.92:0/3393520333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 msgr2=0x7f7e781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 --2- 172.21.15.92:0/3393520333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e781029b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009900 tx=0x7f7e7402a480 comp rx=0 tx=0).stop 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 -- 172.21.15.92:0/3393520333 shutdown_connections 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 --2- 172.21.15.92:0/3393520333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e78102ef0 0x7f7e7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 --2- 172.21.15.92:0/3393520333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e781029b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 --2- 172.21.15.92:0/3393520333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e78100e20 0x7f7e78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 -- 172.21.15.92:0/3393520333 >> 172.21.15.92:0/3393520333 conn(0x7f7e780fc7d0 msgr2=0x7f7e780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 -- 172.21.15.92:0/3393520333 shutdown_connections 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 -- 172.21.15.92:0/3393520333 wait complete. 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 Processor -- start 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.684+0000 7f7e8037e640 1 -- start start 2024-09-11T22:13:49.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e8037e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e78100e20 0x7f7e78199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e8037e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e78199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e8037e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e78102ef0 0x7f7e781a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e8037e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e78110e70 con 0x7f7e78102570 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e8037e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7e78110cf0 con 0x7f7e78100e20 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e8037e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e78110ff0 con 0x7f7e78102ef0 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e78199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e78199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33168/0 (socket says 172.21.15.92:33168) 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 -- 172.21.15.92:0/553581818 learned_addr learned my addr 172.21.15.92:0/553581818 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e78102ef0 0x7f7e781a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e78100e20 0x7f7e78199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 -- 172.21.15.92:0/553581818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e78102ef0 msgr2=0x7f7e781a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e78102ef0 0x7f7e781a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 -- 172.21.15.92:0/553581818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e78100e20 msgr2=0x7f7e78199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e78100e20 0x7f7e78199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 -- 172.21.15.92:0/553581818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e781a4900 con 0x7f7e78102570 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e78102ef0 0x7f7e781a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e7d919640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e78199880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006010 tx=0x7f7e740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.685+0000 7f7e6b7fe640 1 -- 172.21.15.92:0/553581818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7403f030 con 0x7f7e78102570 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.686+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e781a4ad0 con 0x7f7e78102570 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.686+0000 7f7e6b7fe640 1 -- 172.21.15.92:0/553581818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e74003cb0 con 0x7f7e78102570 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.686+0000 7f7e6b7fe640 1 -- 172.21.15.92:0/553581818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74035070 con 0x7f7e78102570 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.686+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e781a4f80 con 0x7f7e78102570 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e6b7fe640 1 -- 172.21.15.92:0/553581818 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7e74042030 con 0x7f7e78102570 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 Processor -- start 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 -- start start 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda36db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda41670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e6b7fe640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7e640a2c10 0x7f7e640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5594cd99d090 con 0x5594cda372f0 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5594cd944a70 con 0x5594cda369d0 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.688+0000 7f7e82757740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594cd999050 con 0x5594cda41bb0 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda41670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7e640a2c10 0x7f7e640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda41670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33174/0 (socket says 172.21.15.92:33174) 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 -- 172.21.15.92:0/2443943797 learned_addr learned my addr 172.21.15.92:0/2443943797 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e6b7fe640 1 -- 172.21.15.92:0/553581818 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7e740efac0 con 0x7f7e78102570 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda36db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7d919640 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 -- 172.21.15.92:0/2443943797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 msgr2=0x5594cda43fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda43fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 -- 172.21.15.92:0/2443943797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 msgr2=0x5594cda36db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda36db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 -- 172.21.15.92:0/2443943797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594cd951130 con 0x5594cda372f0 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7d919640 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7e640a2c10 0x7f7e640a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7e70014670 tx=0x7f7e70014a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda41670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009bc0 tx=0x7f7e6c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda93b1c6c3da36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e68dea640 1 -- 172.21.15.92:0/2443943797 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c016070 con 0x5594cda372f0 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e68dea640 1 -- 172.21.15.92:0/2443943797 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6c011920 con 0x5594cda372f0 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.689+0000 7f7e68dea640 1 -- 172.21.15.92:0/2443943797 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0107f0 con 0x5594cda372f0 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 -- 172.21.15.92:0/2443943797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 msgr2=0x5594cda41670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda41670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009bc0 tx=0x7f7e6c00e6c0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 -- 172.21.15.92:0/2443943797 shutdown_connections 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda41670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 --2- 172.21.15.92:0/2443943797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda36db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 -- 172.21.15.92:0/2443943797 >> 172.21.15.92:0/2443943797 conn(0x5594cda32f60 msgr2=0x5594cda33340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 -- 172.21.15.92:0/2443943797 shutdown_connections 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.690+0000 7f7e82757740 1 -- 172.21.15.92:0/2443943797 wait complete. 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 -- start start 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 Processor -- start 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda41460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda372f0 0x5594cda3e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda41bb0 0x5594cda3ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5594cda44780 con 0x5594cda41bb0 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5594cd957f20 con 0x5594cda369d0 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594cda44900 con 0x5594cda372f0 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e82757740 10 client.0 _ll_register_callbacks cb 0x5594cd921030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda41bb0 0x5594cda3ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda41bb0 0x5594cda3ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33190/0 (socket says 172.21.15.92:33190) 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 -- 172.21.15.92:0/113307315 learned_addr learned my addr 172.21.15.92:0/113307315 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7d919640 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda372f0 0x5594cda3e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda41460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 -- 172.21.15.92:0/113307315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda372f0 msgr2=0x5594cda3e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda372f0 0x5594cda3e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 -- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 msgr2=0x5594cda41460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda41460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.691+0000 7f7e7e91b640 1 -- 172.21.15.92:0/113307315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594cd70e2b0 con 0x5594cda41bb0 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda41bb0 0x5594cda3ecf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e70011dd0 tx=0x7f7e700012a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5870815322e702c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e61ffb640 1 -- 172.21.15.92:0/113307315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70023030 con 0x5594cda41bb0 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e61ffb640 1 -- 172.21.15.92:0/113307315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7e70028410 con 0x5594cda41bb0 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c011920 con 0x5594cda41bb0 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e61ffb640 1 -- 172.21.15.92:0/113307315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e70030050 con 0x5594cda41bb0 2024-09-11T22:13:49.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 10 client.15147 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594cda50010 con 0x5594cda41bb0 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e61ffb640 1 -- 172.21.15.92:0/113307315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6c011920 con 0x5594cda41bb0 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e61ffb640 1 -- 172.21.15.92:0/113307315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f7e7001e040 con 0x5594cda41bb0 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 20 client.15147 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cda526a0 0x5594cda53b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 4 client.15147 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.692+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 --> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594cd9aa600 con 0x5594cda526a0 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.693+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cda526a0 0x5594cda53b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.693+0000 7f7e61ffb640 10 client.15147 ms_handle_connect on v2:172.21.15.110:6838/538708966 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.693+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cda526a0 0x5594cda53b10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62514d5c064a1f3 server_cookie=ef97601e37f4ea91 in_seq=0 out_seq=0 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e61ffb640 1 -- 172.21.15.92:0/113307315 <== mds.0 v2:172.21.15.110:6838/538708966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594cd9aa600 con 0x5594cda526a0 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e61ffb640 10 client.15147 handle_command_reply: tid=0 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 client.15147 shutdown 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda41bb0 msgr2=0x5594cda3ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda41bb0 0x5594cda3ecf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e70011dd0 tx=0x7f7e700012a0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 shutdown_connections 2024-09-11T22:13:49.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cda526a0 0x5594cda53b10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda41bb0 0x5594cda3ecf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda372f0 0x5594cda3e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 --2- 172.21.15.92:0/113307315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda369d0 0x5594cda41460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e61ffb640 0 client.15147 ms_handle_reset on v2:172.21.15.110:6838/538708966 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 >> 172.21.15.92:0/113307315 conn(0x5594cda32f60 msgr2=0x5594cda38c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 shutdown_connections 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 1 -- 172.21.15.92:0/113307315 wait complete. 2024-09-11T22:13:49.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.694+0000 7f7e82757740 20 client.15147 trim_cache size 0 max 16384 2024-09-11T22:13:49.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 Processor -- start 2024-09-11T22:13:49.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 -- start start 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda369d0 0x5594cda33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda372f0 0x5594cda3d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda3d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5594cda51e50 con 0x5594cda369d0 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5594cda52520 con 0x5594cda372f0 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.712+0000 7f7e82757740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594cda52bf0 con 0x5594cda41bb0 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda369d0 0x5594cda33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7d919640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda372f0 0x5594cda3d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda369d0 0x5594cda33340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33198/0 (socket says 172.21.15.92:33198) 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 -- 172.21.15.92:0/3081123508 learned_addr learned my addr 172.21.15.92:0/3081123508 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda3d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 -- 172.21.15.92:0/3081123508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 msgr2=0x5594cda3d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda3d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 -- 172.21.15.92:0/3081123508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda372f0 msgr2=0x5594cda3d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda372f0 0x5594cda3d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 -- 172.21.15.92:0/3081123508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594cda33be0 con 0x5594cda369d0 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7d919640 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda372f0 0x5594cda3d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda369d0 0x5594cda33340 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00ebc0 tx=0x7f7e6c0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3c14aa65f2f321 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e63fff640 1 -- 172.21.15.92:0/3081123508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c02d070 con 0x5594cda369d0 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e63fff640 1 -- 172.21.15.92:0/3081123508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6c009e60 con 0x5594cda369d0 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.713+0000 7f7e63fff640 1 -- 172.21.15.92:0/3081123508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c016040 con 0x5594cda369d0 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 -- 172.21.15.92:0/3081123508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda369d0 msgr2=0x5594cda33340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda369d0 0x5594cda33340 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00ebc0 tx=0x7f7e6c0094e0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 -- 172.21.15.92:0/3081123508 shutdown_connections 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda41bb0 0x5594cda3d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda372f0 0x5594cda3d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 --2- 172.21.15.92:0/3081123508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda369d0 0x5594cda33340 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 -- 172.21.15.92:0/3081123508 >> 172.21.15.92:0/3081123508 conn(0x5594cda32f60 msgr2=0x5594cda3e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 -- 172.21.15.92:0/3081123508 shutdown_connections 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 -- 172.21.15.92:0/3081123508 wait complete. 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 -- start start 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.714+0000 7f7e82757740 1 Processor -- start 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda369d0 0x5594cda5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda5c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda41bb0 0x5594cda55320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5594cda53990 con 0x5594cda372f0 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5594cda532c0 con 0x5594cda41bb0 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594cda54060 con 0x5594cda369d0 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 10 client.0 _ll_register_callbacks cb 0x7f7e7000af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:13:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda5c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda5c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33202/0 (socket says 172.21.15.92:33202) 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 -- 172.21.15.92:0/1397907081 learned_addr learned my addr 172.21.15.92:0/1397907081 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda41bb0 0x5594cda55320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda369d0 0x5594cda5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 -- 172.21.15.92:0/1397907081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda369d0 msgr2=0x5594cda5bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda369d0 0x5594cda5bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 -- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda41bb0 msgr2=0x5594cda55320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda41bb0 0x5594cda55320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 -- 172.21.15.92:0/1397907081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594cda55b30 con 0x5594cda372f0 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda369d0 0x5594cda5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7e91b640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda41bb0 0x5594cda55320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e7d919640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda5c4e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e740ed140 tx=0x7f7e740ed590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66676c47864f9474 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e61ffb640 1 -- 172.21.15.92:0/1397907081 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74049070 con 0x5594cda372f0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e61ffb640 1 -- 172.21.15.92:0/1397907081 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7e740b94b0 con 0x5594cda372f0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.715+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c009e60 con 0x5594cda372f0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e61ffb640 1 -- 172.21.15.92:0/1397907081 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e740b96b0 con 0x5594cda372f0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594cda5a5c0 con 0x5594cda372f0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e61ffb640 1 -- 172.21.15.92:0/1397907081 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6c009e60 con 0x5594cda372f0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e61ffb640 1 -- 172.21.15.92:0/1397907081 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f7e740bc0c0 con 0x5594cda372f0 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 20 client.15153 populate_metadata read hostname 'smithi092' 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cdac1460 0x5594cdac0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:49.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 4 client.15153 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 --> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] -- command(tid 0: {"prefix": "status"}) -- 0x5594cd9a4b10 con 0x5594cdac1460 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.716+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cdac1460 0x5594cdac0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e7e11a640 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cdac1460 0x5594cdac0170 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53aa4a4c533ffc08 server_cookie=4a522e2609601bd1 in_seq=0 out_seq=0 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e61ffb640 10 client.15153 ms_handle_connect on v2:172.21.15.110:6838/538708966 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e61ffb640 1 -- 172.21.15.92:0/1397907081 <== mds.0 v2:172.21.15.110:6838/538708966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5594cd9a4b10 con 0x5594cdac1460 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e61ffb640 10 client.15153 handle_command_reply: tid=0 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 client.15153 shutdown 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 msgr2=0x5594cda5c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda5c4e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e740ed140 tx=0x7f7e740ed590 comp rx=0 tx=0).stop 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 shutdown_connections 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x5594cdac1460 0x5594cdac0170 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5594cda41bb0 0x5594cda55320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5594cda372f0 0x5594cda5c4e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e61ffb640 0 client.15153 ms_handle_reset on v2:172.21.15.110:6838/538708966 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 --2- 172.21.15.92:0/1397907081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594cda369d0 0x5594cda5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 >> 172.21.15.92:0/1397907081 conn(0x5594cda32f60 msgr2=0x5594cda395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 shutdown_connections 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 1 -- 172.21.15.92:0/1397907081 wait complete. 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.717+0000 7f7e82757740 20 client.15153 trim_cache size 0 max 16384 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-09-11T22:13:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24455, 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 105.952527266, 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 116.76404648800001 2024-09-11T22:13:49.729 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7e640a2c10 msgr2=0x7f7e640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7e640a2c10 0x7f7e640a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7e70014670 tx=0x7f7e70014a70 comp rx=0 tx=0).stop 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 msgr2=0x7f7e78199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e78199880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006010 tx=0x7f7e740042f0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 shutdown_connections 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f7e640a2c10 0x7f7e640a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e78102ef0 0x7f7e781a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e78102570 0x7f7e78199880 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 --2- 172.21.15.92:0/553581818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e78100e20 0x7f7e78199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 >> 172.21.15.92:0/553581818 conn(0x7f7e780fc7d0 msgr2=0x7f7e780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 shutdown_connections 2024-09-11T22:13:49.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:49.718+0000 7f7e8037e640 1 -- 172.21.15.92:0/553581818 wait complete. 2024-09-11T22:13:50.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:49 smithi110 ceph-mon[34830]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:13:50.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:49 smithi092 ceph-mon[31173]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:13:50.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:49 smithi153 ceph-mon[35045]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:13:50.438 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T22:13:50.438 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T22:13:50.449 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T22:13:50.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T22:13:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 -- 172.21.15.92:0/2176771558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 msgr2=0x7f3060103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 --2- 172.21.15.92:0/2176771558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060103280 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3054009900 tx=0x7f305402a410 comp rx=0 tx=0).stop 2024-09-11T22:13:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 -- 172.21.15.92:0/2176771558 shutdown_connections 2024-09-11T22:13:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 --2- 172.21.15.92:0/2176771558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30601037c0 0x7f306010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 --2- 172.21.15.92:0/2176771558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060103280 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 --2- 172.21.15.92:0/2176771558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3060101c00 0x7f3060102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 -- 172.21.15.92:0/2176771558 >> 172.21.15.92:0/2176771558 conn(0x7f30600fd390 msgr2=0x7f30600ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 -- 172.21.15.92:0/2176771558 shutdown_connections 2024-09-11T22:13:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.192+0000 7f3064f53640 1 -- 172.21.15.92:0/2176771558 wait complete. 2024-09-11T22:13:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 Processor -- start 2024-09-11T22:13:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 -- start start 2024-09-11T22:13:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3060101c00 0x7f3060197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30601037c0 0x7f306019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f306010d710 con 0x7f3060102e00 2024-09-11T22:13:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f306010d590 con 0x7f3060101c00 2024-09-11T22:13:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f3064f53640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f306010d890 con 0x7f30601037c0 2024-09-11T22:13:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33210/0 (socket says 172.21.15.92:33210) 2024-09-11T22:13:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 -- 172.21.15.92:0/2396411182 learned_addr learned my addr 172.21.15.92:0/2396411182 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305e59c640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3060101c00 0x7f3060197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 -- 172.21.15.92:0/2396411182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30601037c0 msgr2=0x7f306019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30601037c0 0x7f306019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 -- 172.21.15.92:0/2396411182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3060101c00 msgr2=0x7f3060197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3060101c00 0x7f3060197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.193+0000 7f305dd9b640 1 -- 172.21.15.92:0/2396411182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306019fa50 con 0x7f3060102e00 2024-09-11T22:13:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.194+0000 7f305dd9b640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060198360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3054005940 tx=0x7f30540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.194+0000 7f304f7fe640 1 -- 172.21.15.92:0/2396411182 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305403f030 con 0x7f3060102e00 2024-09-11T22:13:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.194+0000 7f304f7fe640 1 -- 172.21.15.92:0/2396411182 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3054003cb0 con 0x7f3060102e00 2024-09-11T22:13:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.194+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306019fce0 con 0x7f3060102e00 2024-09-11T22:13:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.194+0000 7f304f7fe640 1 -- 172.21.15.92:0/2396411182 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054035070 con 0x7f3060102e00 2024-09-11T22:13:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.194+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30601a01c0 con 0x7f3060102e00 2024-09-11T22:13:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.195+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30601090c0 con 0x7f3060102e00 2024-09-11T22:13:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.197+0000 7f304f7fe640 1 -- 172.21.15.92:0/2396411182 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3054042030 con 0x7f3060102e00 2024-09-11T22:13:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.197+0000 7f304f7fe640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f30480a2cc0 0x7f30480a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.197+0000 7f305e59c640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f30480a2cc0 0x7f30480a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.197+0000 7f304f7fe640 1 -- 172.21.15.92:0/2396411182 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f30540f0130 con 0x7f3060102e00 2024-09-11T22:13:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.198+0000 7f305e59c640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f30480a2cc0 0x7f30480a5180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3050005d00 tx=0x7f3050005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.201+0000 7f304f7fe640 1 -- 172.21.15.92:0/2396411182 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30540bcbd0 con 0x7f3060102e00 2024-09-11T22:13:51.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.388+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3060062630 con 0x7f30480a2cc0 2024-09-11T22:13:51.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.390+0000 7f304f7fe640 1 -- 172.21.15.92:0/2396411182 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3060062630 con 0x7f30480a2cc0 2024-09-11T22:13:51.391 INFO:teuthology.orchestra.run.smithi092.stdout:Backend: cephadm 2024-09-11T22:13:51.391 INFO:teuthology.orchestra.run.smithi092.stdout:Available: Yes 2024-09-11T22:13:51.391 INFO:teuthology.orchestra.run.smithi092.stdout:Paused: No 2024-09-11T22:13:51.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f30480a2cc0 msgr2=0x7f30480a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:51.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f30480a2cc0 0x7f30480a5180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3050005d00 tx=0x7f3050005c50 comp rx=0 tx=0).stop 2024-09-11T22:13:51.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 msgr2=0x7f3060198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:51.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060198360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3054005940 tx=0x7f30540042f0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 shutdown_connections 2024-09-11T22:13:51.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f30480a2cc0 0x7f30480a5180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30601037c0 0x7f306019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3060102e00 0x7f3060198360 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 --2- 172.21.15.92:0/2396411182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3060101c00 0x7f3060197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:51.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 >> 172.21.15.92:0/2396411182 conn(0x7f30600fd390 msgr2=0x7f3060106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:51.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 shutdown_connections 2024-09-11T22:13:51.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:51.392+0000 7f3064f53640 1 -- 172.21.15.92:0/2396411182 wait complete. 2024-09-11T22:13:52.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T22:13:52.410 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:52.841 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:52 smithi092 ceph-mon[31173]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:52.841 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:52 smithi092 ceph-mon[31173]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:13:52.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 -- 172.21.15.92:0/3974776083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc103d30 msgr2=0x7f04fc106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:52.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 --2- 172.21.15.92:0/3974776083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc103d30 0x7f04fc106120 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a410 comp rx=0 tx=0).stop 2024-09-11T22:13:52.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 -- 172.21.15.92:0/3974776083 shutdown_connections 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 --2- 172.21.15.92:0/3974776083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04fc1067f0 0x7f04fc108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 --2- 172.21.15.92:0/3974776083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc103d30 0x7f04fc106120 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 --2- 172.21.15.92:0/3974776083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04fc101370 0x7f04fc103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 -- 172.21.15.92:0/3974776083 >> 172.21.15.92:0/3974776083 conn(0x7f04fc0fb610 msgr2=0x7f04fc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 -- 172.21.15.92:0/3974776083 shutdown_connections 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.918+0000 7f05016f2640 1 -- 172.21.15.92:0/3974776083 wait complete. 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 Processor -- start 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 -- start start 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc101370 0x7f04fc10b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04fc103d30 0x7f04fc109af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04fc1067f0 0x7f04fc10a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d7e0 con 0x7f04fc101370 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d660 con 0x7f04fc103d30 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f05016f2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d960 con 0x7f04fc1067f0 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.919+0000 7f04faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc101370 0x7f04fc10b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc101370 0x7f04fc10b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33228/0 (socket says 172.21.15.92:33228) 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 -- 172.21.15.92:0/3776654898 learned_addr learned my addr 172.21.15.92:0/3776654898 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04fb7fe640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04fc1067f0 0x7f04fc10a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04fa7fc640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04fc103d30 0x7f04fc109af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 -- 172.21.15.92:0/3776654898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04fc1067f0 msgr2=0x7f04fc10a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04fc1067f0 0x7f04fc10a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 -- 172.21.15.92:0/3776654898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04fc103d30 msgr2=0x7f04fc109af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04fc103d30 0x7f04fc109af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 -- 172.21.15.92:0/3776654898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc10a8f0 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04fa7fc640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04fc103d30 0x7f04fc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04fb7fe640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04fc1067f0 0x7f04fc10a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04faffd640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc101370 0x7f04fc10b4a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00c970 tx=0x7f04ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04e3fff640 1 -- 172.21.15.92:0/3776654898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec007bb0 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc1a1fc0 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04e3fff640 1 -- 172.21.15.92:0/3776654898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04ec007d50 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f04e3fff640 1 -- 172.21.15.92:0/3776654898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00f450 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.920+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1a24f0 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.921+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc0fd100 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.922+0000 7f04e3fff640 1 -- 172.21.15.92:0/3776654898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f04ec004eb0 con 0x7f04fc101370 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.923+0000 7f04e3fff640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04e40a2c10 0x7f04e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.923+0000 7f04fa7fc640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04e40a2c10 0x7f04e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.923+0000 7f04e3fff640 1 -- 172.21.15.92:0/3776654898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f04ec0cf300 con 0x7f04fc101370 2024-09-11T22:13:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.924+0000 7f04fa7fc640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04e40a2c10 0x7f04e40a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f04f002a930 tx=0x7f04f0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:52.927+0000 7f04e3fff640 1 -- 172.21.15.92:0/3776654898 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04ec014030 con 0x7f04fc101370 2024-09-11T22:13:53.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:52 smithi153 ceph-mon[35045]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:53.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:52 smithi153 ceph-mon[35045]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:13:53.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.102+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f04fc062630 con 0x7f04e40a2c10 2024-09-11T22:13:53.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.116+0000 7f04e3fff640 1 -- 172.21.15.92:0/3776654898 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f04fc062630 con 0x7f04e40a2c10 2024-09-11T22:13:53.117 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T22:13:53.117 INFO:teuthology.orchestra.run.smithi092.stdout:mds.a smithi092 running (2m) 52s ago 2m 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 77b3d4333b9d 2024-09-11T22:13:53.117 INFO:teuthology.orchestra.run.smithi092.stdout:mds.b smithi110 running (2m) 53s ago 2m 25.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 94775968d4fb 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.c smithi153 running (2m) 53s ago 2m 17.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 a592d442b3a6 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.d smithi092 running (2m) 52s ago 2m 14.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 6576448ede5a 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.e smithi110 running (2m) 53s ago 2m 16.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 e28c89905fea 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.f smithi153 running (2m) 53s ago 2m 19.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 9c7c2db94ea0 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.g smithi092 running (2m) 52s ago 2m 15.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 8f461e252924 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.h smithi110 running (2m) 53s ago 2m 17.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 40db97aa5302 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.i smithi153 running (2m) 53s ago 2m 17.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 f9f398948c24 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.j smithi092 running (2m) 52s ago 2m 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 6535b833ed09 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.k smithi110 running (2m) 53s ago 2m 18.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 de5280b9f56a 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mds.l smithi153 running (2m) 53s ago 2m 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 d1a8ee70b076 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.x smithi092 *:9283,8765 running (9m) 52s ago 9m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 708a65413e49 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.y smithi110 *:8443,8765 running (7m) 53s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 ab1b1fe2c08b 2024-09-11T22:13:53.118 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.z smithi153 *:8443,8765 running (7m) 53s ago 7m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 eb4416760230 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a smithi092 running (9m) 52s ago 9m 52.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 966a88dcdfc4 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b smithi110 running (7m) 53s ago 7m 39.6M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 8130dd59e95e 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c smithi153 running (8m) 53s ago 8m 39.9M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 4f85c6c52807 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (7m) 52s ago 7m 55.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 64447f5f06c1 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi110 running (7m) 53s ago 6m 48.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2a830a3e651b 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi153 running (6m) 53s ago 6m 59.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 bd9ab63edc21 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi092 running (6m) 52s ago 6m 54.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2f5398a17833 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi110 running (5m) 53s ago 5m 42.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 a3fadcc2760d 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi153 running (5m) 53s ago 5m 58.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 22c2a138371f 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 smithi092 running (4m) 52s ago 4m 57.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 de39e20a52f0 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 smithi110 running (4m) 53s ago 4m 55.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 45265d510f72 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.8 smithi153 running (4m) 53s ago 4m 57.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 9c33413ac49e 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.9 smithi092 running (3m) 52s ago 3m 53.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 49b82df131f1 2024-09-11T22:13:53.119 INFO:teuthology.orchestra.run.smithi092.stdout:osd.10 smithi110 running (3m) 53s ago 3m 43.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 342dc30183da 2024-09-11T22:13:53.120 INFO:teuthology.orchestra.run.smithi092.stdout:osd.11 smithi153 running (2m) 53s ago 2m 43.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 7b7c1e39f234 2024-09-11T22:13:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04e40a2c10 msgr2=0x7f04e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04e40a2c10 0x7f04e40a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f04f002a930 tx=0x7f04f0005fd0 comp rx=0 tx=0).stop 2024-09-11T22:13:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc101370 msgr2=0x7f04fc10b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc101370 0x7f04fc10b4a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00c970 tx=0x7f04ec00ce30 comp rx=0 tx=0).stop 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 shutdown_connections 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f04e40a2c10 0x7f04e40a50d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04fc1067f0 0x7f04fc10a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04fc103d30 0x7f04fc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 --2- 172.21.15.92:0/3776654898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04fc101370 0x7f04fc10b4a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 >> 172.21.15.92:0/3776654898 conn(0x7f04fc0fb610 msgr2=0x7f04fc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 shutdown_connections 2024-09-11T22:13:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:53.118+0000 7f05016f2640 1 -- 172.21.15.92:0/3776654898 wait complete. 2024-09-11T22:13:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:52 smithi110 ceph-mon[34830]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:52 smithi110 ceph-mon[34830]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:13:53.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T22:13:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:54.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 -- 172.21.15.92:0/1212012538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2788102e00 msgr2=0x7f2788103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 --2- 172.21.15.92:0/1212012538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2788102e00 0x7f2788103280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2778009900 tx=0x7f277802a410 comp rx=0 tx=0).stop 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 -- 172.21.15.92:0/1212012538 shutdown_connections 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 --2- 172.21.15.92:0/1212012538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27881037c0 0x7f278810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 --2- 172.21.15.92:0/1212012538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2788102e00 0x7f2788103280 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 --2- 172.21.15.92:0/1212012538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2788101c00 0x7f2788102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 -- 172.21.15.92:0/1212012538 >> 172.21.15.92:0/1212012538 conn(0x7f27880fd390 msgr2=0x7f27880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 -- 172.21.15.92:0/1212012538 shutdown_connections 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.571+0000 7f27904bb640 1 -- 172.21.15.92:0/1212012538 wait complete. 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 Processor -- start 2024-09-11T22:13:54.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 -- start start 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2788101c00 0x7f2788197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2788102e00 0x7f2788198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27881037c0 0x7f278819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f278810d730 con 0x7f27881037c0 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f278810d5b0 con 0x7f2788102e00 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f27904bb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f278810d8b0 con 0x7f2788101c00 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278ea58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27881037c0 0x7f278819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278ea58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27881037c0 0x7f278819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33232/0 (socket says 172.21.15.92:33232) 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278ea58640 1 -- 172.21.15.92:0/3225269923 learned_addr learned my addr 172.21.15.92:0/3225269923 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278e257640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2788101c00 0x7f2788197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278da56640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2788102e00 0x7f2788198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278ea58640 1 -- 172.21.15.92:0/3225269923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2788101c00 msgr2=0x7f2788197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:54.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278ea58640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2788101c00 0x7f2788197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.572+0000 7f278ea58640 1 -- 172.21.15.92:0/3225269923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2788102e00 msgr2=0x7f2788198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f278ea58640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2788102e00 0x7f2788198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f278ea58640 1 -- 172.21.15.92:0/3225269923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278819fa40 con 0x7f27881037c0 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f278da56640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2788102e00 0x7f2788198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f278ea58640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27881037c0 0x7f278819f3d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f278000eaa0 tx=0x7f278000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f277f7fe640 1 -- 172.21.15.92:0/3225269923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278000cdb0 con 0x7f27881037c0 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278819fd30 con 0x7f27881037c0 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f277f7fe640 1 -- 172.21.15.92:0/3225269923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2780005020 con 0x7f27881037c0 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f277f7fe640 1 -- 172.21.15.92:0/3225269923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800116d0 con 0x7f27881037c0 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.573+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27881a0210 con 0x7f27881037c0 2024-09-11T22:13:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.574+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27881090a0 con 0x7f27881037c0 2024-09-11T22:13:54.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.575+0000 7f277f7fe640 1 -- 172.21.15.92:0/3225269923 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f27800051d0 con 0x7f27881037c0 2024-09-11T22:13:54.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.576+0000 7f277f7fe640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27740a2c40 0x7f27740a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:54.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.576+0000 7f278e257640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27740a2c40 0x7f27740a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:54.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.576+0000 7f277f7fe640 1 -- 172.21.15.92:0/3225269923 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f27800cf6e0 con 0x7f27881037c0 2024-09-11T22:13:54.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.576+0000 7f278e257640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27740a2c40 0x7f27740a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2788102c60 tx=0x7f278400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:54.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.580+0000 7f277f7fe640 1 -- 172.21.15.92:0/3225269923 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2780016030 con 0x7f27881037c0 2024-09-11T22:13:54.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.756+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2788062630 con 0x7f27740a2c40 2024-09-11T22:13:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.763+0000 7f277f7fe640 1 -- 172.21.15.92:0/3225269923 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2788062630 con 0x7f27740a2c40 2024-09-11T22:13:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T22:13:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:mds.all 12/12 55s ago 2m smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-11T22:13:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:mgr 3/3 55s ago 7m smithi092=x;smithi110=y;smithi153=z;count:3 2024-09-11T22:13:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:mon 3/3 55s ago 8m smithi092:172.21.15.92=a;smithi110:172.21.15.110=b;smithi153:172.21.15.153=c;count:3 2024-09-11T22:13:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:osd 12 55s ago - 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.765+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27740a2c40 msgr2=0x7f27740a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.765+0000 7f27904bb640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27740a2c40 0x7f27740a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2788102c60 tx=0x7f278400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27881037c0 msgr2=0x7f278819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27881037c0 0x7f278819f3d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f278000eaa0 tx=0x7f278000ef60 comp rx=0 tx=0).stop 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 shutdown_connections 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f27740a2c40 0x7f27740a5100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27881037c0 0x7f278819f3d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2788102e00 0x7f2788198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 --2- 172.21.15.92:0/3225269923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2788101c00 0x7f2788197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 >> 172.21.15.92:0/3225269923 conn(0x7f27880fd390 msgr2=0x7f2788106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 shutdown_connections 2024-09-11T22:13:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:54.766+0000 7f27904bb640 1 -- 172.21.15.92:0/3225269923 wait complete. 2024-09-11T22:13:55.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:54 smithi092 ceph-mon[31173]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:55.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:54 smithi092 ceph-mon[31173]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:55.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:54 smithi153 ceph-mon[35045]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:54 smithi153 ceph-mon[35045]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:54 smithi110 ceph-mon[34830]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:54 smithi110 ceph-mon[34830]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:13:55.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T22:13:55.798 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:56.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:55 smithi153 ceph-mon[35045]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:56.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:55 smithi092 ceph-mon[31173]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:56.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:55 smithi110 ceph-mon[34830]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 -- 172.21.15.92:0/2516326853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc100ad0 msgr2=0x7fb5bc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 --2- 172.21.15.92:0/2516326853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc100ad0 0x7fb5bc100ed0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009900 tx=0x7fb5b802a410 comp rx=0 tx=0).stop 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 -- 172.21.15.92:0/2516326853 shutdown_connections 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 --2- 172.21.15.92:0/2516326853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5bc102690 0x7fb5bc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 --2- 172.21.15.92:0/2516326853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5bc101cd0 0x7fb5bc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 --2- 172.21.15.92:0/2516326853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc100ad0 0x7fb5bc100ed0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 -- 172.21.15.92:0/2516326853 >> 172.21.15.92:0/2516326853 conn(0x7fb5bc0fc280 msgr2=0x7fb5bc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 -- 172.21.15.92:0/2516326853 shutdown_connections 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.276+0000 7fb5c432d640 1 -- 172.21.15.92:0/2516326853 wait complete. 2024-09-11T22:13:56.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 Processor -- start 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 -- start start 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5bc100ad0 0x7fb5bc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5bc101cd0 0x7fb5bc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc102690 0x7fb5bc19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10c660 con 0x7fb5bc102690 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10c4e0 con 0x7fb5bc100ad0 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c432d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10c7e0 con 0x7fb5bc101cd0 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c28ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc102690 0x7fb5bc19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c28ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc102690 0x7fb5bc19f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32918/0 (socket says 172.21.15.92:32918) 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c28ca640 1 -- 172.21.15.92:0/472106589 learned_addr learned my addr 172.21.15.92:0/472106589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c20c9640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5bc100ad0 0x7fb5bc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.277+0000 7fb5c18c8640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5bc101cd0 0x7fb5bc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c28ca640 1 -- 172.21.15.92:0/472106589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5bc101cd0 msgr2=0x7fb5bc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c28ca640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5bc101cd0 0x7fb5bc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c28ca640 1 -- 172.21.15.92:0/472106589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5bc100ad0 msgr2=0x7fb5bc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c28ca640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5bc100ad0 0x7fb5bc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c28ca640 1 -- 172.21.15.92:0/472106589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc19fb80 con 0x7fb5bc102690 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c28ca640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc102690 0x7fb5bc19f510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400cad0 tx=0x7fb5b400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5b37fe640 1 -- 172.21.15.92:0/472106589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400ad60 con 0x7fb5bc102690 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc19fe70 con 0x7fb5bc102690 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5b37fe640 1 -- 172.21.15.92:0/472106589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5b4005020 con 0x7fb5bc102690 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5b37fe640 1 -- 172.21.15.92:0/472106589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400f6b0 con 0x7fb5bc102690 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.278+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1a03a0 con 0x7fb5bc102690 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.279+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc1a0600 con 0x7fb5bc102690 2024-09-11T22:13:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.280+0000 7fb5b37fe640 1 -- 172.21.15.92:0/472106589 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb5b40051d0 con 0x7fb5bc102690 2024-09-11T22:13:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.281+0000 7fb5b37fe640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb5a80a2c40 0x7fb5a80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.281+0000 7fb5c20c9640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb5a80a2c40 0x7fb5a80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.281+0000 7fb5b37fe640 1 -- 172.21.15.92:0/472106589 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb5b40cf4e0 con 0x7fb5bc102690 2024-09-11T22:13:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.282+0000 7fb5c20c9640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb5a80a2c40 0x7fb5a80a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.285+0000 7fb5b37fe640 1 -- 172.21.15.92:0/472106589 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5b4014030 con 0x7fb5bc102690 2024-09-11T22:13:56.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.463+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb5bc1a07d0 con 0x7fb5a80a2c40 2024-09-11T22:13:56.466 INFO:teuthology.orchestra.run.smithi092.stdout:HOST ADDR LABELS STATUS 2024-09-11T22:13:56.466 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 172.21.15.92 2024-09-11T22:13:56.466 INFO:teuthology.orchestra.run.smithi092.stdout:smithi110 172.21.15.110 2024-09-11T22:13:56.466 INFO:teuthology.orchestra.run.smithi092.stdout:smithi153 172.21.15.153 2024-09-11T22:13:56.466 INFO:teuthology.orchestra.run.smithi092.stdout:3 hosts in cluster 2024-09-11T22:13:56.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.464+0000 7fb5b37fe640 1 -- 172.21.15.92:0/472106589 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb5bc1a07d0 con 0x7fb5a80a2c40 2024-09-11T22:13:56.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb5a80a2c40 msgr2=0x7fb5a80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:56.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb5a80a2c40 0x7fb5a80a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc102690 msgr2=0x7fb5bc19f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:56.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc102690 0x7fb5bc19f510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400cad0 tx=0x7fb5b400ced0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 shutdown_connections 2024-09-11T22:13:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb5a80a2c40 0x7fb5a80a5100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc102690 0x7fb5bc19f510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5bc101cd0 0x7fb5bc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 --2- 172.21.15.92:0/472106589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5bc100ad0 0x7fb5bc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 >> 172.21.15.92:0/472106589 conn(0x7fb5bc0fc280 msgr2=0x7fb5bc069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 shutdown_connections 2024-09-11T22:13:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:56.467+0000 7fb5c432d640 1 -- 172.21.15.92:0/472106589 wait complete. 2024-09-11T22:13:57.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:56 smithi153 ceph-mon[35045]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:57.118 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:56 smithi110 ceph-mon[34830]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:57.132 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:56 smithi092 ceph-mon[31173]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:13:57.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T22:13:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:57.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.927+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2751144479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 msgr2=0x7f0d80100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:57.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.927+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2751144479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80100ed0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009900 tx=0x7f0d7002a410 comp rx=0 tx=0).stop 2024-09-11T22:13:57.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2751144479 shutdown_connections 2024-09-11T22:13:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2751144479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d80102690 0x7f0d80108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2751144479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d80101cd0 0x7f0d80102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2751144479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80100ed0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2751144479 >> 172.21.15.92:0/2751144479 conn(0x7f0d800fc280 msgr2=0x7f0d800fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2751144479 shutdown_connections 2024-09-11T22:13:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2751144479 wait complete. 2024-09-11T22:13:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 Processor -- start 2024-09-11T22:13:57.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- start start 2024-09-11T22:13:57.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:57.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d80101cd0 0x7f0d801983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:57.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d80102690 0x7f0d8019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:57.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d6f0 con 0x7f0d80100ad0 2024-09-11T22:13:57.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d570 con 0x7f0d80102690 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.928+0000 7f0d85c3d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d870 con 0x7f0d80101cd0 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32946/0 (socket says 172.21.15.92:32946) 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 -- 172.21.15.92:0/2417641372 learned_addr learned my addr 172.21.15.92:0/2417641372 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7ffff640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d80102690 0x7f0d8019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 -- 172.21.15.92:0/2417641372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d80101cd0 msgr2=0x7f0d801983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7effd640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d80101cd0 0x7f0d801983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d80101cd0 0x7f0d801983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 -- 172.21.15.92:0/2417641372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d80102690 msgr2=0x7f0d8019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d80102690 0x7f0d8019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 -- 172.21.15.92:0/2417641372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8019fad0 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7ffff640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d80102690 0x7f0d8019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7effd640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d80101cd0 0x7f0d801983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7f7fe640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80197ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d700314e0 tx=0x7f0d700042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7cff9640 1 -- 172.21.15.92:0/2417641372 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7003f030 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7cff9640 1 -- 172.21.15.92:0/2417641372 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d70003cb0 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8019fd60 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d7cff9640 1 -- 172.21.15.92:0/2417641372 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70035070 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.929+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a01d0 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.930+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d80108010 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.932+0000 7f0d7cff9640 1 -- 172.21.15.92:0/2417641372 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0d70042030 con 0x7f0d80100ad0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.932+0000 7f0d7cff9640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d680a2cc0 0x7f0d680a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.932+0000 7f0d7effd640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d680a2cc0 0x7f0d680a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:57.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.932+0000 7f0d7cff9640 1 -- 172.21.15.92:0/2417641372 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0d700eff50 con 0x7f0d80100ad0 2024-09-11T22:13:57.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.933+0000 7f0d7effd640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d680a2cc0 0x7f0d680a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d801993c0 tx=0x7f0d74005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:57.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:57.935+0000 7f0d7cff9640 1 -- 172.21.15.92:0/2417641372 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d700bc9f0 con 0x7f0d80100ad0 2024-09-11T22:13:58.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:57 smithi153 ceph-mon[35045]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:58.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.109+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 --> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0d80062630 con 0x7f0d680a2cc0 2024-09-11T22:13:58.111 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:57 smithi092 ceph-mon[31173]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:58.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.112+0000 7f0d7cff9640 1 -- 172.21.15.92:0/2417641372 <== mgr.14150 v2:172.21.15.92:6800/1619835932 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0d80062630 con 0x7f0d680a2cc0 2024-09-11T22:13:58.113 INFO:teuthology.orchestra.run.smithi092.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T22:13:58.113 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T22:13:58.113 INFO:teuthology.orchestra.run.smithi092.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-11T22:13:58.113 INFO:teuthology.orchestra.run.smithi092.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T22:13:58.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d680a2cc0 msgr2=0x7f0d680a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:58.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d680a2cc0 0x7f0d680a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d801993c0 tx=0x7f0d74005e50 comp rx=0 tx=0).stop 2024-09-11T22:13:58.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 msgr2=0x7f0d80197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:58.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80197ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d700314e0 tx=0x7f0d700042e0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 shutdown_connections 2024-09-11T22:13:58.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f0d680a2cc0 0x7f0d680a5180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d80102690 0x7f0d8019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d80101cd0 0x7f0d801983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 --2- 172.21.15.92:0/2417641372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d80100ad0 0x7f0d80197ea0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 >> 172.21.15.92:0/2417641372 conn(0x7f0d800fc280 msgr2=0x7f0d801051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:58.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 shutdown_connections 2024-09-11T22:13:58.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:58.115+0000 7f0d85c3d640 1 -- 172.21.15.92:0/2417641372 wait complete. 2024-09-11T22:13:58.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:57 smithi110 ceph-mon[34830]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:58.826 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:58 smithi092 ceph-mon[31173]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:58.853 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T22:13:58.862 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T22:13:58.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T22:13:59.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:58 smithi153 ceph-mon[35045]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:59.105 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:58 smithi110 ceph-mon[34830]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:13:59.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 -- 172.21.15.92:0/380425733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4101c00 msgr2=0x7f78a4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:59.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 --2- 172.21.15.92:0/380425733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4101c00 0x7f78a4102000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009900 tx=0x7f78a002a410 comp rx=0 tx=0).stop 2024-09-11T22:13:59.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 -- 172.21.15.92:0/380425733 shutdown_connections 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 --2- 172.21.15.92:0/380425733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41037c0 0x7f78a410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 --2- 172.21.15.92:0/380425733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4102e00 0x7f78a4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 --2- 172.21.15.92:0/380425733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4101c00 0x7f78a4102000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 -- 172.21.15.92:0/380425733 >> 172.21.15.92:0/380425733 conn(0x7f78a40fd390 msgr2=0x7f78a40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 -- 172.21.15.92:0/380425733 shutdown_connections 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.591+0000 7f78abbcd640 1 -- 172.21.15.92:0/380425733 wait complete. 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 Processor -- start 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 -- start start 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4101c00 0x7f78a419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4102e00 0x7f78a419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41037c0 0x7f78a41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78a410d760 con 0x7f78a4102e00 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78a410d5e0 con 0x7f78a41037c0 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78abbcd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78a410d8e0 con 0x7f78a4101c00 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78a9168640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4102e00 0x7f78a419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78a9168640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4102e00 0x7f78a419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32964/0 (socket says 172.21.15.92:32964) 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78a9168640 1 -- 172.21.15.92:0/3767327960 learned_addr learned my addr 172.21.15.92:0/3767327960 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.592+0000 7f78aa16a640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41037c0 0x7f78a41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78a9969640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4101c00 0x7f78a419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78a9168640 1 -- 172.21.15.92:0/3767327960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4101c00 msgr2=0x7f78a419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78a9168640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4101c00 0x7f78a419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78a9168640 1 -- 172.21.15.92:0/3767327960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41037c0 msgr2=0x7f78a41a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78a9168640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41037c0 0x7f78a41a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78a9168640 1 -- 172.21.15.92:0/3767327960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a41a1cb0 con 0x7f78a4102e00 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78aa16a640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41037c0 0x7f78a41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78a9168640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4102e00 0x7f78a419a5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f789400d770 tx=0x7f789400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f789affd640 1 -- 172.21.15.92:0/3767327960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894017070 con 0x7f78a4102e00 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f789affd640 1 -- 172.21.15.92:0/3767327960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78940054d0 con 0x7f78a4102e00 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f789affd640 1 -- 172.21.15.92:0/3767327960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894004cf0 con 0x7f78a4102e00 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a41a1fa0 con 0x7f78a4102e00 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.593+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41a2400 con 0x7f78a4102e00 2024-09-11T22:13:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.594+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a41a2630 con 0x7f78a4102e00 2024-09-11T22:13:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.596+0000 7f789affd640 1 -- 172.21.15.92:0/3767327960 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7894004e90 con 0x7f78a4102e00 2024-09-11T22:13:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.596+0000 7f789affd640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f78900a2c40 0x7f78900a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.597+0000 7f78a9969640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f78900a2c40 0x7f78900a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.597+0000 7f789affd640 1 -- 172.21.15.92:0/3767327960 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f78940cf340 con 0x7f78a4102e00 2024-09-11T22:13:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.597+0000 7f78a9969640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f78900a2c40 0x7f78900a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f78a002a930 tx=0x7f78a0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.600+0000 7f789affd640 1 -- 172.21.15.92:0/3767327960 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f789409be80 con 0x7f78a4102e00 2024-09-11T22:13:59.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.802+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f78a41a29c0 con 0x7f78a4102e00 2024-09-11T22:13:59.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.803+0000 7f789affd640 1 -- 172.21.15.92:0/3767327960 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3134 (secure 0 0 0) 0x7f789409b780 con 0x7f78a4102e00 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:e27 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:btime 2024-09-11T22:12:40:147657+0000 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-09-11T22:13:59.805 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 27 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-09-11T22:12:03.420066+0000 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-09-11T22:12:40.147644+0000 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 300 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 0 2024-09-11T22:13:59.806 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 5 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1,2,3,4 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=24455,1=24451,2=24433,3=24428,4=24427} 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3,4] 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data disabled 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-09-11T22:13:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 0 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 24455 members: 24455,24451,24433,24428,24427 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b{0:24455} state up:active seq 4 addr [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.e{0:24401} state up:standby-replay seq 1 addr [v2:172.21.15.110:6834/1455978865,v1:172.21.15.110:6835/1455978865] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.i{1:24451} state up:active seq 14 addr [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.l{1:24400} state up:standby-replay seq 1 addr [v2:172.21.15.153:6832/490387556,v1:172.21.15.153:6833/490387556] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c{2:24433} state up:active seq 16 addr [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.k{2:24377} state up:standby-replay seq 1 addr [v2:172.21.15.110:6832/797341789,v1:172.21.15.110:6833/797341789] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.h{3:24428} state up:active seq 16 addr [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a{3:14619} state up:standby-replay seq 1 addr [v2:172.21.15.92:6840/2477024139,v1:172.21.15.92:6841/2477024139] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.f{4:24427} state up:active seq 17 addr [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.808 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.g{4:14607} state up:standby-replay seq 1 addr [v2:172.21.15.92:6838/1790524059,v1:172.21.15.92:6839/1790524059] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.809 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:59.809 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:59.809 INFO:teuthology.orchestra.run.smithi092.stdout:Standby daemons: 2024-09-11T22:13:59.809 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:13:59.809 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.92:6834/1683671615,v1:172.21.15.92:6835/1683671615] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.809 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.92:6836/2477546203,v1:172.21.15.92:6837/2477546203] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:13:59.813 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-09-11T22:13:59.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f78900a2c40 msgr2=0x7f78900a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:59.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f78900a2c40 0x7f78900a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f78a002a930 tx=0x7f78a0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4102e00 msgr2=0x7f78a419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:59.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4102e00 0x7f78a419a5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f789400d770 tx=0x7f789400dc30 comp rx=0 tx=0).stop 2024-09-11T22:13:59.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 shutdown_connections 2024-09-11T22:13:59.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f78900a2c40 0x7f78900a5100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41037c0 0x7f78a41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a4102e00 0x7f78a419a5c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 --2- 172.21.15.92:0/3767327960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4101c00 0x7f78a419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:59.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 >> 172.21.15.92:0/3767327960 conn(0x7f78a40fd390 msgr2=0x7f78a4108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:59.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 shutdown_connections 2024-09-11T22:13:59.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:13:59.805+0000 7f78abbcd640 1 -- 172.21.15.92:0/3767327960 wait complete. 2024-09-11T22:14:00.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:13:59 smithi153 ceph-mon[35045]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:00.155 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:13:59 smithi092 ceph-mon[31173]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:00.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:13:59 smithi110 ceph-mon[34830]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:00.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T22:14:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:00.953 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:00 smithi092 ceph-mon[31173]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:00.953 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:00 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3767327960' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:14:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:00 smithi153 ceph-mon[35045]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:01.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:00 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3767327960' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:14:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:00 smithi110 ceph-mon[34830]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:00 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3767327960' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 -- 172.21.15.92:0/3680306864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14104f90 msgr2=0x7fed14107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 --2- 172.21.15.92:0/3680306864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14104f90 0x7fed14107380 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fed08009900 tx=0x7fed0802a410 comp rx=0 tx=0).stop 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 -- 172.21.15.92:0/3680306864 shutdown_connections 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 --2- 172.21.15.92:0/3680306864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed141078c0 0x7fed14109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 --2- 172.21.15.92:0/3680306864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14104f90 0x7fed14107380 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 --2- 172.21.15.92:0/3680306864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed14102660 0x7fed14104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 -- 172.21.15.92:0/3680306864 >> 172.21.15.92:0/3680306864 conn(0x7fed140fc280 msgr2=0x7fed140fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 -- 172.21.15.92:0/3680306864 shutdown_connections 2024-09-11T22:14:01.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.319+0000 7fed18880640 1 -- 172.21.15.92:0/3680306864 wait complete. 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 Processor -- start 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 -- start start 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14102660 0x7fed14197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14104f90 0x7fed14198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed141078c0 0x7fed1419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410d710 con 0x7fed14102660 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410d590 con 0x7fed14104f90 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed18880640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410d890 con 0x7fed141078c0 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed1259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14102660 0x7fed14197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed1259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14102660 0x7fed14197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32972/0 (socket says 172.21.15.92:32972) 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed1259c640 1 -- 172.21.15.92:0/3183636710 learned_addr learned my addr 172.21.15.92:0/3183636710 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed11d9b640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14104f90 0x7fed14198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.320+0000 7fed12d9d640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed141078c0 0x7fed1419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed1259c640 1 -- 172.21.15.92:0/3183636710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed141078c0 msgr2=0x7fed1419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed1259c640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed141078c0 0x7fed1419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed1259c640 1 -- 172.21.15.92:0/3183636710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14104f90 msgr2=0x7fed14198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed1259c640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14104f90 0x7fed14198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed1259c640 1 -- 172.21.15.92:0/3183636710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1419fa80 con 0x7fed14102660 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed1259c640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14102660 0x7fed14197e50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fed0400e9f0 tx=0x7fed0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed037fe640 1 -- 172.21.15.92:0/3183636710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0400cd80 con 0x7fed14102660 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed141004b0 con 0x7fed14102660 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed037fe640 1 -- 172.21.15.92:0/3183636710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed040054d0 con 0x7fed14102660 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed037fe640 1 -- 172.21.15.92:0/3183636710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04010490 con 0x7fed14102660 2024-09-11T22:14:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.321+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141009c0 con 0x7fed14102660 2024-09-11T22:14:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.322+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed14105370 con 0x7fed14102660 2024-09-11T22:14:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.323+0000 7fed037fe640 1 -- 172.21.15.92:0/3183636710 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fed04005000 con 0x7fed14102660 2024-09-11T22:14:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.324+0000 7fed037fe640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fecfc0a2c40 0x7fecfc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.324+0000 7fed11d9b640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fecfc0a2c40 0x7fecfc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.324+0000 7fed037fe640 1 -- 172.21.15.92:0/3183636710 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fed040cfa60 con 0x7fed14102660 2024-09-11T22:14:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.325+0000 7fed11d9b640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fecfc0a2c40 0x7fecfc0a5100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed08004770 tx=0x7fed08000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.327+0000 7fed037fe640 1 -- 172.21.15.92:0/3183636710 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed040541b0 con 0x7fed14102660 2024-09-11T22:14:01.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.521+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fed140fdd70 con 0x7fed14102660 2024-09-11T22:14:01.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.522+0000 7fed037fe640 1 -- 172.21.15.92:0/3183636710 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5596 (secure 0 0 0) 0x7fed140fdd70 con 0x7fed14102660 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 92 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:fsid a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-09-11T22:04:29.869322+0000 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-09-11T22:13:16.772187+0000 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:crush_version 27 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:full_ratio 0.95 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:backfillfull_ratio 0.9 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:nearfull_ratio 0.85 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:require_min_compat_client luminous 2024-09-11T22:14:01.524 INFO:teuthology.orchestra.run.smithi092.stdout:min_compat_client luminous 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:require_osd_release squid 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:stretch_mode_enabled false 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:max_osd 12 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6802/2928117998,v1:172.21.15.92:6803/2928117998] [v2:172.21.15.92:6804/2928117998,v1:172.21.15.92:6805/2928117998] exists,up 4595b0bf-4ae7-499d-98e9-3aa88e41ff70 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/1945048542,v1:172.21.15.110:6801/1945048542] [v2:172.21.15.110:6802/1945048542,v1:172.21.15.110:6803/1945048542] exists,up 26c935bd-8f25-4bbd-bdcf-8d55523c3e34 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6800/2033986544,v1:172.21.15.153:6801/2033986544] [v2:172.21.15.153:6802/2033986544,v1:172.21.15.153:6803/2033986544] exists,up 1be9b69c-ef19-405e-9f42-0964e338d4c9 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 up in weight 1 up_from 27 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6810/931675439,v1:172.21.15.92:6811/931675439] [v2:172.21.15.92:6812/931675439,v1:172.21.15.92:6813/931675439] exists,up 0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 up in weight 1 up_from 35 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/712194146,v1:172.21.15.110:6809/712194146] [v2:172.21.15.110:6810/712194146,v1:172.21.15.110:6811/712194146] exists,up b9039a81-37d0-4a40-8050-394638ea88a5 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 up in weight 1 up_from 40 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6808/1354584850,v1:172.21.15.153:6809/1354584850] [v2:172.21.15.153:6810/1354584850,v1:172.21.15.153:6811/1354584850] exists,up a0472991-0835-438f-9579-8e10646de923 2024-09-11T22:14:01.525 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/433608050,v1:172.21.15.92:6819/433608050] [v2:172.21.15.92:6820/433608050,v1:172.21.15.92:6821/433608050] exists,up 647ec3ad-3258-47f3-9c07-f242cda2ec4d 2024-09-11T22:14:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/1905314527,v1:172.21.15.110:6817/1905314527] [v2:172.21.15.110:6818/1905314527,v1:172.21.15.110:6819/1905314527] exists,up 5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb 2024-09-11T22:14:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6816/979097601,v1:172.21.15.153:6817/979097601] [v2:172.21.15.153:6818/979097601,v1:172.21.15.153:6819/979097601] exists,up 37cde37b-22c1-4525-9d66-ca50a29c7060 2024-09-11T22:14:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:osd.9 up in weight 1 up_from 66 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6826/3782209439,v1:172.21.15.92:6827/3782209439] [v2:172.21.15.92:6828/3782209439,v1:172.21.15.92:6829/3782209439] exists,up c38dd0d9-39fb-4c67-9958-18eeb82ef314 2024-09-11T22:14:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:osd.10 up in weight 1 up_from 71 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/3399897751,v1:172.21.15.110:6825/3399897751] [v2:172.21.15.110:6826/3399897751,v1:172.21.15.110:6827/3399897751] exists,up abd86e0e-ad63-46dd-961a-54b939ad22b9 2024-09-11T22:14:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:osd.11 up in weight 1 up_from 78 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6824/3726820847,v1:172.21.15.153:6825/3726820847] [v2:172.21.15.153:6826/3726820847,v1:172.21.15.153:6827/3726820847] exists,up 74de56f5-f0f9-44d6-ac7a-c7edacd28a4a 2024-09-11T22:14:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.b [2,1] 2024-09-11T22:14:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.f [8,11] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.14 [2,4] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.13 [4,9] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.1e [3,9] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.3c [4,9] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 4.b [9,10] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 4.e [5,8] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 4.13 [9,8] 2024-09-11T22:14:01.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 4.1f [9,8] 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2458669467 expires 2024-09-12T22:05:15.603416+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2872938890 expires 2024-09-12T22:05:15.603416+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/3563316301 expires 2024-09-12T22:05:15.603416+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/733069068 expires 2024-09-12T22:05:15.603416+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/3468435378 expires 2024-09-12T22:04:52.791314+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/125981315 expires 2024-09-12T22:04:52.791314+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/1519869556 expires 2024-09-12T22:04:52.791314+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/1519869556 expires 2024-09-12T22:04:52.791314+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/733069068 expires 2024-09-12T22:05:15.603416+0000 2024-09-11T22:14:01.528 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/4186575934 expires 2024-09-12T22:04:52.791314+0000 2024-09-11T22:14:01.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fecfc0a2c40 msgr2=0x7fecfc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:01.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fecfc0a2c40 0x7fecfc0a5100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed08004770 tx=0x7fed08000bc0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14102660 msgr2=0x7fed14197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:01.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14102660 0x7fed14197e50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fed0400e9f0 tx=0x7fed0400edf0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 shutdown_connections 2024-09-11T22:14:01.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fecfc0a2c40 0x7fecfc0a5100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed141078c0 0x7fed1419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14104f90 0x7fed14198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 --2- 172.21.15.92:0/3183636710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed14102660 0x7fed14197e50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:01.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 >> 172.21.15.92:0/3183636710 conn(0x7fed140fc280 msgr2=0x7fed14102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:01.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 shutdown_connections 2024-09-11T22:14:01.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:01.525+0000 7fed18880640 1 -- 172.21.15.92:0/3183636710 wait complete. 2024-09-11T22:14:02.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:01 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:02.042 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:01 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3183636710' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:14:02.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:01 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:02.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:01 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3183636710' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:14:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:01 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:02.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:01 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3183636710' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:14:02.864 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T22:14:02.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:14:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:02 smithi110 ceph-mon[34830]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:02 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:02 smithi092 ceph-mon[31173]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:02 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:02 smithi153 ceph-mon[35045]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:02 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 -- 172.21.15.92:0/11030893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4470101df0 msgr2=0x7f44701021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 --2- 172.21.15.92:0/11030893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4470101df0 0x7f44701021d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4460009ca0 tx=0x7f446002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 -- 172.21.15.92:0/11030893 shutdown_connections 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 --2- 172.21.15.92:0/11030893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470106f20 0x7f447010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 --2- 172.21.15.92:0/11030893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44701027a0 0x7f44701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 --2- 172.21.15.92:0/11030893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4470101df0 0x7f44701021d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 -- 172.21.15.92:0/11030893 >> 172.21.15.92:0/11030893 conn(0x7f44700fd8f0 msgr2=0x7f44700ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 -- 172.21.15.92:0/11030893 shutdown_connections 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 -- 172.21.15.92:0/11030893 wait complete. 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 Processor -- start 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.636+0000 7f4475375640 1 -- start start 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4475375640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470101df0 0x7f4470198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4475375640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44701027a0 0x7f4470198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4475375640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4470106f20 0x7f447019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4475375640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4470111ed0 con 0x7f44701027a0 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4475375640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4470111d50 con 0x7f4470101df0 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4475375640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4470112050 con 0x7f4470106f20 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44701027a0 0x7f4470198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44701027a0 0x7f4470198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32992/0 (socket says 172.21.15.92:32992) 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 -- 172.21.15.92:0/3658793773 learned_addr learned my addr 172.21.15.92:0/3658793773 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446effd640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470101df0 0x7f4470198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446f7fe640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4470106f20 0x7f447019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 -- 172.21.15.92:0/3658793773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4470106f20 msgr2=0x7f447019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4470106f20 0x7f447019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 -- 172.21.15.92:0/3658793773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470101df0 msgr2=0x7f4470198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470101df0 0x7f4470198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 -- 172.21.15.92:0/3658793773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447019d400 con 0x7f44701027a0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f446e7fc640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44701027a0 0x7f4470198990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f445800b4a0 tx=0x7f445800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4467fff640 1 -- 172.21.15.92:0/3658793773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458014070 con 0x7f44701027a0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4467fff640 1 -- 172.21.15.92:0/3658793773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4458010ac0 con 0x7f44701027a0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447019d6f0 con 0x7f44701027a0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.637+0000 7f4467fff640 1 -- 172.21.15.92:0/3658793773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458017c30 con 0x7f44701027a0 2024-09-11T22:14:03.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.638+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44701a4fd0 con 0x7f44701027a0 2024-09-11T22:14:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.638+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44701021d0 con 0x7f44701027a0 2024-09-11T22:14:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.640+0000 7f4467fff640 1 -- 172.21.15.92:0/3658793773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4458017070 con 0x7f44701027a0 2024-09-11T22:14:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.640+0000 7f4467fff640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f445c0a2c10 0x7f445c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.640+0000 7f446effd640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f445c0a2c10 0x7f445c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.640+0000 7f4467fff640 1 -- 172.21.15.92:0/3658793773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f44580ce810 con 0x7f44701027a0 2024-09-11T22:14:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.641+0000 7f446effd640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f445c0a2c10 0x7f445c0a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4460009a60 tx=0x7f4460031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.644+0000 7f4467fff640 1 -- 172.21.15.92:0/3658793773 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f445809b3c0 con 0x7f44701027a0 2024-09-11T22:14:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.863+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4470109b70 con 0x7f44701027a0 2024-09-11T22:14:03.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.864+0000 7f4467fff640 1 -- 172.21.15.92:0/3658793773 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11980 (secure 0 0 0) 0x7f445809acc0 con 0x7f44701027a0 2024-09-11T22:14:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:14:03.865 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"btime":"2024-09-11T22:12:40:147657+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:40.147644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:14:03.866 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-09-11T22:14:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f445c0a2c10 msgr2=0x7f445c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f445c0a2c10 0x7f445c0a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4460009a60 tx=0x7f4460031000 comp rx=0 tx=0).stop 2024-09-11T22:14:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44701027a0 msgr2=0x7f4470198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44701027a0 0x7f4470198990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f445800b4a0 tx=0x7f445800b960 comp rx=0 tx=0).stop 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 shutdown_connections 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f445c0a2c10 0x7f445c0a50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4470106f20 0x7f447019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44701027a0 0x7f4470198990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 --2- 172.21.15.92:0/3658793773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470101df0 0x7f4470198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 >> 172.21.15.92:0/3658793773 conn(0x7f44700fd8f0 msgr2=0x7f44700fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 shutdown_connections 2024-09-11T22:14:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:03.867+0000 7f4475375640 1 -- 172.21.15.92:0/3658793773 wait complete. 2024-09-11T22:14:04.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:14:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:04 smithi092 ceph-mon[31173]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:04 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3658793773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:14:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:05.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:04 smithi153 ceph-mon[35045]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:04 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3658793773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:14:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:05.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:04 smithi110 ceph-mon[34830]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:04 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3658793773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:14:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:05.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:14:05.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 -- 172.21.15.92:0/2077289794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c102de0 msgr2=0x7ff36c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 --2- 172.21.15.92:0/2077289794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c102de0 0x7ff36c1031c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009900 tx=0x7ff35c02a480 comp rx=0 tx=0).stop 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 -- 172.21.15.92:0/2077289794 shutdown_connections 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 --2- 172.21.15.92:0/2077289794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff36c10c180 0x7ff36c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 --2- 172.21.15.92:0/2077289794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c103700 0x7ff36c10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 --2- 172.21.15.92:0/2077289794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c102de0 0x7ff36c1031c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 -- 172.21.15.92:0/2077289794 >> 172.21.15.92:0/2077289794 conn(0x7ff36c0fc7a0 msgr2=0x7ff36c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 -- 172.21.15.92:0/2077289794 shutdown_connections 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.429+0000 7ff3742cd640 1 -- 172.21.15.92:0/2077289794 wait complete. 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 Processor -- start 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 -- start start 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff36c102de0 0x7ff36c100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c103700 0x7ff36c100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c10c180 0x7ff36c1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c111e60 con 0x7ff36c103700 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c111ce0 con 0x7ff36c10c180 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff3742cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c111fe0 con 0x7ff36c102de0 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff371868640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c103700 0x7ff36c100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff371868640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c103700 0x7ff36c100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38554/0 (socket says 172.21.15.92:38554) 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff371868640 1 -- 172.21.15.92:0/3385751837 learned_addr learned my addr 172.21.15.92:0/3385751837 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff37286a640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c10c180 0x7ff36c1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.430+0000 7ff372069640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff36c102de0 0x7ff36c100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff371868640 1 -- 172.21.15.92:0/3385751837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff36c102de0 msgr2=0x7ff36c100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff371868640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff36c102de0 0x7ff36c100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff371868640 1 -- 172.21.15.92:0/3385751837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c10c180 msgr2=0x7ff36c1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff371868640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c10c180 0x7ff36c1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff371868640 1 -- 172.21.15.92:0/3385751837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c1a6840 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff372069640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff36c102de0 0x7ff36c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff371868640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c103700 0x7ff36c100f80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff36800d950 tx=0x7ff36800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff3637fe640 1 -- 172.21.15.92:0/3385751837 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368017070 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff3637fe640 1 -- 172.21.15.92:0/3385751837 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff368009c30 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36c1a6b30 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff3637fe640 1 -- 172.21.15.92:0/3385751837 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36800f620 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.431+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff36c1a7070 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.432+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36c1031c0 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.433+0000 7ff3637fe640 1 -- 172.21.15.92:0/3385751837 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff36800f7c0 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.434+0000 7ff3637fe640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3580a2c10 0x7ff3580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.434+0000 7ff372069640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3580a2c10 0x7ff3580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.434+0000 7ff3637fe640 1 -- 172.21.15.92:0/3385751837 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff3680cf1c0 con 0x7ff36c103700 2024-09-11T22:14:05.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.434+0000 7ff372069640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3580a2c10 0x7ff3580a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff35c031440 tx=0x7ff35c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:05.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.438+0000 7ff3637fe640 1 -- 172.21.15.92:0/3385751837 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff36809bc60 con 0x7ff36c103700 2024-09-11T22:14:05.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.637+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff36c107240 con 0x7ff36c103700 2024-09-11T22:14:05.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.638+0000 7ff3637fe640 1 -- 172.21.15.92:0/3385751837 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11980 (secure 0 0 0) 0x7ff368009de0 con 0x7ff36c103700 2024-09-11T22:14:05.715 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:14:05.716 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"btime":"2024-09-11T22:12:40:147657+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:12:40.147644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:14:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-09-11T22:14:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3580a2c10 msgr2=0x7ff3580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3580a2c10 0x7ff3580a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff35c031440 tx=0x7ff35c005a20 comp rx=0 tx=0).stop 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c103700 msgr2=0x7ff36c100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c103700 0x7ff36c100f80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff36800d950 tx=0x7ff36800de10 comp rx=0 tx=0).stop 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 shutdown_connections 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff3580a2c10 0x7ff3580a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c10c180 0x7ff36c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff36c103700 0x7ff36c100f80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 --2- 172.21.15.92:0/3385751837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff36c102de0 0x7ff36c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 >> 172.21.15.92:0/3385751837 conn(0x7ff36c0fc7a0 msgr2=0x7ff36c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 shutdown_connections 2024-09-11T22:14:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:05.640+0000 7ff3742cd640 1 -- 172.21.15.92:0/3385751837 wait complete. 2024-09-11T22:14:05.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:05 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3385751837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:14:06.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:05 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3385751837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:14:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:05 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3385751837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:14:06.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:14:06.697 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:07.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:06 smithi092 ceph-mon[31173]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:07.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:06 smithi153 ceph-mon[35045]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:07.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 -- 172.21.15.92:0/1775287522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000101400 msgr2=0x7f700010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 --2- 172.21.15.92:0/1775287522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000101400 0x7f700010d930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009cd0 tx=0x7f6ff802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 -- 172.21.15.92:0/1775287522 shutdown_connections 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 --2- 172.21.15.92:0/1775287522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000101400 0x7f700010d930 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 --2- 172.21.15.92:0/1775287522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7000100a60 0x7f7000100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 --2- 172.21.15.92:0/1775287522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7000106a60 0x7f7000106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 -- 172.21.15.92:0/1775287522 >> 172.21.15.92:0/1775287522 conn(0x7f70000fc780 msgr2=0x7f70000feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 -- 172.21.15.92:0/1775287522 shutdown_connections 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.150+0000 7f7008095640 1 -- 172.21.15.92:0/1775287522 wait complete. 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 Processor -- start 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 -- start start 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7000100a60 0x7f7000102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7000101400 0x7f7000102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000106a60 0x7f70001030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7000110d10 con 0x7f7000106a60 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7000110b90 con 0x7f7000101400 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7008095640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7000110e90 con 0x7f7000100a60 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000106a60 0x7f70001030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000106a60 0x7f70001030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38570/0 (socket says 172.21.15.92:38570) 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 -- 172.21.15.92:0/3828568586 learned_addr learned my addr 172.21.15.92:0/3828568586 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7005e31640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7000100a60 0x7f7000102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7005630640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7000101400 0x7f7000102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 -- 172.21.15.92:0/3828568586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7000100a60 msgr2=0x7f7000102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7000100a60 0x7f7000102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 -- 172.21.15.92:0/3828568586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7000101400 msgr2=0x7f7000102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7000101400 0x7f7000102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.151+0000 7f7006632640 1 -- 172.21.15.92:0/3828568586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70001a6840 con 0x7f7000106a60 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.152+0000 7f7006632640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000106a60 0x7f70001030b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8005cc0 tx=0x7f6ff8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.152+0000 7f6ff6ffd640 1 -- 172.21.15.92:0/3828568586 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8004450 con 0x7f7000106a60 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.152+0000 7f6ff6ffd640 1 -- 172.21.15.92:0/3828568586 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ff8031c90 con 0x7f7000106a60 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.152+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70001a6ad0 con 0x7f7000106a60 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.152+0000 7f6ff6ffd640 1 -- 172.21.15.92:0/3828568586 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff803b400 con 0x7f7000106a60 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.152+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70001a7010 con 0x7f7000106a60 2024-09-11T22:14:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.153+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7000100ec0 con 0x7f7000106a60 2024-09-11T22:14:07.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.154+0000 7f6ff6ffd640 1 -- 172.21.15.92:0/3828568586 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6ff8004680 con 0x7f7000106a60 2024-09-11T22:14:07.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.155+0000 7f6ff6ffd640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6fec0a2c10 0x7f6fec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.155+0000 7f7005e31640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6fec0a2c10 0x7f6fec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.155+0000 7f6ff6ffd640 1 -- 172.21.15.92:0/3828568586 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6ff802e030 con 0x7f7000106a60 2024-09-11T22:14:07.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.155+0000 7f7005e31640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6fec0a2c10 0x7f6fec0a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005fd0 tx=0x7f6ffc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:07.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.158+0000 7f6ff6ffd640 1 -- 172.21.15.92:0/3828568586 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ff80bd660 con 0x7f7000106a60 2024-09-11T22:14:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:06 smithi110 ceph-mon[34830]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.324+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70001a7240 con 0x7f7000106a60 2024-09-11T22:14:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.325+0000 7f6ff6ffd640 1 -- 172.21.15.92:0/3828568586 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24737 (secure 0 0 0) 0x7f6ff80bcf60 con 0x7f7000106a60 2024-09-11T22:14:07.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:14:07.327 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:13:16.772187+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:14:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6fec0a2c10 msgr2=0x7f6fec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6fec0a2c10 0x7f6fec0a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005fd0 tx=0x7f6ffc00b390 comp rx=0 tx=0).stop 2024-09-11T22:14:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000106a60 msgr2=0x7f70001030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000106a60 0x7f70001030b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8005cc0 tx=0x7f6ff8003c20 comp rx=0 tx=0).stop 2024-09-11T22:14:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 shutdown_connections 2024-09-11T22:14:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f6fec0a2c10 0x7f6fec0a50d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7000106a60 0x7f70001030b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7000101400 0x7f7000102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 --2- 172.21.15.92:0/3828568586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7000100a60 0x7f7000102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 >> 172.21.15.92:0/3828568586 conn(0x7f70000fc780 msgr2=0x7f700006a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 shutdown_connections 2024-09-11T22:14:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:07.327+0000 7f7008095640 1 -- 172.21.15.92:0/3828568586 wait complete. 2024-09-11T22:14:07.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:07 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3828568586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:14:07.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:12:03.420066+0000', 'modified': '2024-09-11T22:12:40.147644+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24455, 'mds_1': 24451, 'mds_2': 24433, 'mds_3': 24428, 'mds_4': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14607': {'gid': 14607, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6839/1790524059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1790524059}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1790524059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14619': {'gid': 14619, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6841/2477024139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6840', 'nonce': 2477024139}, {'type': 'v1', 'addr': '172.21.15.92:6841', 'nonce': 2477024139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6833/797341789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 797341789}, {'type': 'v1', 'addr': '172.21.15.110:6833', 'nonce': 797341789}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6833/490387556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 490387556}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 490387556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/1455978865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 1455978865}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 1455978865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.153:6835/1638990868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 1638990868}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 1638990868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.110:6837/3571280995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 3571280995}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 3571280995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.153:6837/1434479445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 1434479445}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 1434479445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.153:6839/229095273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 229095273}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 229095273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.110:6839/538708966', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 538708966}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 538708966}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24455, 'qdb_cluster': [24455, 24451, 24433, 24428, 24427]} 2024-09-11T22:14:07.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:14:07.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-11T22:14:08.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:07 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3828568586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:14:08.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:07 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3828568586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:14:08.245 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.633+0000 7f5454688640 1 -- 172.21.15.92:0/2503587842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c106a80 msgr2=0x7f544c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.633+0000 7f5454688640 1 --2- 172.21.15.92:0/2503587842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c106a80 0x7f544c106e60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5448009930 tx=0x7f544802a420 comp rx=0 tx=0).stop 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.633+0000 7f5454688640 1 -- 172.21.15.92:0/2503587842 shutdown_connections 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.633+0000 7f5454688640 1 --2- 172.21.15.92:0/2503587842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c101420 0x7f544c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.633+0000 7f5454688640 1 --2- 172.21.15.92:0/2503587842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f544c100a80 0x7f544c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.633+0000 7f5454688640 1 --2- 172.21.15.92:0/2503587842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c106a80 0x7f544c106e60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.633+0000 7f5454688640 1 -- 172.21.15.92:0/2503587842 >> 172.21.15.92:0/2503587842 conn(0x7f544c0fc7a0 msgr2=0x7f544c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.634+0000 7f5454688640 1 -- 172.21.15.92:0/2503587842 shutdown_connections 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.634+0000 7f5454688640 1 -- 172.21.15.92:0/2503587842 wait complete. 2024-09-11T22:14:08.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.634+0000 7f5454688640 1 Processor -- start 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.634+0000 7f5454688640 1 -- start start 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5454688640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c100a80 0x7f544c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5454688640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c101420 0x7f544c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5454688640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f544c106a80 0x7f544c1a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5454688640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f544c110d30 con 0x7f544c100a80 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5454688640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f544c110bb0 con 0x7f544c101420 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5454688640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f544c110eb0 con 0x7f544c106a80 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c100a80 0x7f544c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c100a80 0x7f544c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38598/0 (socket says 172.21.15.92:38598) 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 -- 172.21.15.92:0/4242475610 learned_addr learned my addr 172.21.15.92:0/4242475610 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:08.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452c25640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f544c106a80 0x7f544c1a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 -- 172.21.15.92:0/4242475610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f544c106a80 msgr2=0x7f544c1a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f544c106a80 0x7f544c1a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 -- 172.21.15.92:0/4242475610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c101420 msgr2=0x7f544c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5451c23640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c101420 0x7f544c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c101420 0x7f544c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 -- 172.21.15.92:0/4242475610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c1a4780 con 0x7f544c100a80 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5452424640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c100a80 0x7f544c1991c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5448005940 tx=0x7f54480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f54437fe640 1 -- 172.21.15.92:0/4242475610 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544803f030 con 0x7f544c100a80 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544c1a4950 con 0x7f544c100a80 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.635+0000 7f54437fe640 1 -- 172.21.15.92:0/4242475610 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5448003cb0 con 0x7f544c100a80 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.636+0000 7f54437fe640 1 -- 172.21.15.92:0/4242475610 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448035070 con 0x7f544c100a80 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.636+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544c1a4da0 con 0x7f544c100a80 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.638+0000 7f54437fe640 1 -- 172.21.15.92:0/4242475610 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5448042030 con 0x7f544c100a80 2024-09-11T22:14:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.638+0000 7f5456a61740 1 Processor -- start 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.638+0000 7f5456a61740 1 -- start start 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5456a61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fafdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5456a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5456a61740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fbcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5456a61740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fef8f16090 con 0x55fef8faf9d0 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5456a61740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fef8ebda70 con 0x55fef8fbabb0 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5456a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fef8f12050 con 0x55fef8fb02f0 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fafdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fafdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38614/0 (socket says 172.21.15.92:38614) 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 -- 172.21.15.92:0/3232660747 learned_addr learned my addr 172.21.15.92:0/3232660747 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f54437fe640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f54380a2c10 0x7f54380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5451c23640 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fbcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 -- 172.21.15.92:0/3232660747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 msgr2=0x55fef8fba670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f54437fe640 1 -- 172.21.15.92:0/4242475610 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f54480efbd0 con 0x7f544c100a80 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452424640 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fba670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 -- 172.21.15.92:0/3232660747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 msgr2=0x55fef8fbcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fbcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 -- 172.21.15.92:0/3232660747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef8eca130 con 0x55fef8faf9d0 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5451c23640 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fbcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452424640 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f54380a2c10 0x7f54380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.639+0000 7f5452c25640 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fafdb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f54440073a0 tx=0x7f544400e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3949a2613d6192bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:08.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5440ff9640 1 -- 172.21.15.92:0/3232660747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544401fe60 con 0x55fef8faf9d0 2024-09-11T22:14:08.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5452c25640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f54380a2c10 0x7f54380a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5444023000 tx=0x7f5444020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:08.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5440ff9640 1 -- 172.21.15.92:0/3232660747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f544401e8f0 con 0x55fef8faf9d0 2024-09-11T22:14:08.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5440ff9640 1 -- 172.21.15.92:0/3232660747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544401d780 con 0x55fef8faf9d0 2024-09-11T22:14:08.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5456a61740 1 -- 172.21.15.92:0/3232660747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 msgr2=0x55fef8fafdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5456a61740 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fafdb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f54440073a0 tx=0x7f544400e7e0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5456a61740 1 -- 172.21.15.92:0/3232660747 shutdown_connections 2024-09-11T22:14:08.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5456a61740 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fbcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5456a61740 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5456a61740 1 --2- 172.21.15.92:0/3232660747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fafdb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.640+0000 7f5456a61740 1 -- 172.21.15.92:0/3232660747 >> 172.21.15.92:0/3232660747 conn(0x55fef8fabf60 msgr2=0x55fef8fac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:08.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 -- 172.21.15.92:0/3232660747 shutdown_connections 2024-09-11T22:14:08.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 -- 172.21.15.92:0/3232660747 wait complete. 2024-09-11T22:14:08.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 -- start start 2024-09-11T22:14:08.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 Processor -- start 2024-09-11T22:14:08.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fb02f0 0x55fef8fb7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fbabb0 0x55fef8fb7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fbd780 con 0x55fef8fbabb0 2024-09-11T22:14:08.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fef8ed0f20 con 0x55fef8fb02f0 2024-09-11T22:14:08.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fbd900 con 0x55fef8faf9d0 2024-09-11T22:14:08.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5456a61740 10 client.0 _ll_register_callbacks cb 0x55fef8e9a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5452c25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fbabb0 0x55fef8fb7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5452c25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fbabb0 0x55fef8fb7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38618/0 (socket says 172.21.15.92:38618) 2024-09-11T22:14:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5452c25640 1 -- 172.21.15.92:0/2661439235 learned_addr learned my addr 172.21.15.92:0/2661439235 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5452424640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.641+0000 7f5451c23640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fb02f0 0x55fef8fb7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5452c25640 1 -- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 msgr2=0x55fef8fba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5452c25640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5452c25640 1 -- 172.21.15.92:0/2661439235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fb02f0 msgr2=0x55fef8fb7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5452c25640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fb02f0 0x55fef8fb7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5452c25640 1 -- 172.21.15.92:0/2661439235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef8c872b0 con 0x55fef8fbabb0 2024-09-11T22:14:08.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5452424640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:14:08.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5452c25640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fbabb0 0x55fef8fb7c60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f544401e670 tx=0x7f544401e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a8ead57d0081b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:08.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5435ffb640 1 -- 172.21.15.92:0/2661439235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544401fe60 con 0x55fef8fbabb0 2024-09-11T22:14:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5435ffb640 1 -- 172.21.15.92:0/2661439235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5444025070 con 0x55fef8fbabb0 2024-09-11T22:14:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544401e8f0 con 0x55fef8fbabb0 2024-09-11T22:14:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5435ffb640 1 -- 172.21.15.92:0/2661439235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5444007a50 con 0x55fef8fbabb0 2024-09-11T22:14:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5456a61740 10 client.15225 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fef8fc8fd0 con 0x55fef8fbabb0 2024-09-11T22:14:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.642+0000 7f5435ffb640 1 -- 172.21.15.92:0/2661439235 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f544401e8f0 con 0x55fef8fbabb0 2024-09-11T22:14:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5435ffb640 1 -- 172.21.15.92:0/2661439235 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f54440295c0 con 0x55fef8fbabb0 2024-09-11T22:14:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5456a61740 10 client.15225 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5456a61740 10 client.15225 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T22:14:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5456a61740 20 client.15225 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5456a61740 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef8fcb430 0x55fef8fcc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5456a61740 4 client.15225 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:14:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 --> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fef8f23600 con 0x55fef8fcb430 2024-09-11T22:14:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5452424640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef8fcb430 0x55fef8fcc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5435ffb640 10 client.15225 ms_handle_connect on v2:172.21.15.153:6834/1638990868 2024-09-11T22:14:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.643+0000 7f5452424640 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef8fcb430 0x55fef8fcc8a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef143d15f71fb1bb server_cookie=d361a3c4110a0bf8 in_seq=0 out_seq=0 2024-09-11T22:14:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5435ffb640 1 -- 172.21.15.92:0/2661439235 <== mds.4 v2:172.21.15.153:6834/1638990868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fef8f23600 con 0x55fef8fcb430 2024-09-11T22:14:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5435ffb640 10 client.15225 handle_command_reply: tid=0 2024-09-11T22:14:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 client.15225 shutdown 2024-09-11T22:14:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fbabb0 msgr2=0x55fef8fb7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fbabb0 0x55fef8fb7c60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f544401e670 tx=0x7f544401e6a0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 shutdown_connections 2024-09-11T22:14:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef8fcb430 0x55fef8fcc8a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fbabb0 0x55fef8fb7c60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5435ffb640 0 client.15225 ms_handle_reset on v2:172.21.15.153:6834/1638990868 2024-09-11T22:14:08.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fb02f0 0x55fef8fb7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 --2- 172.21.15.92:0/2661439235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 >> 172.21.15.92:0/2661439235 conn(0x55fef8fabf60 msgr2=0x55fef8fb1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:08.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 shutdown_connections 2024-09-11T22:14:08.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.644+0000 7f5456a61740 1 -- 172.21.15.92:0/2661439235 wait complete. 2024-09-11T22:14:08.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.645+0000 7f5456a61740 20 client.15225 trim_cache size 0 max 16384 2024-09-11T22:14:08.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 Processor -- start 2024-09-11T22:14:08.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 -- start start 2024-09-11T22:14:08.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fc7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fb02f0 0x55fef8fd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fcabe0 con 0x55fef8fb02f0 2024-09-11T22:14:08.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fcb2b0 con 0x55fef8fbabb0 2024-09-11T22:14:08.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.667+0000 7f5456a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fcb980 con 0x55fef8faf9d0 2024-09-11T22:14:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fb02f0 0x55fef8fd4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fb02f0 0x55fef8fd4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38624/0 (socket says 172.21.15.92:38624) 2024-09-11T22:14:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 -- 172.21.15.92:0/671229753 learned_addr learned my addr 172.21.15.92:0/671229753 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5452c25640 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 -- 172.21.15.92:0/671229753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 msgr2=0x55fef8fc7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fc7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 -- 172.21.15.92:0/671229753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 msgr2=0x55fef8fd50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 -- 172.21.15.92:0/671229753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef8faf750 con 0x55fef8fb02f0 2024-09-11T22:14:08.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5452c25640 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:08.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5451c23640 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fb02f0 0x55fef8fd4b80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f543c0058c0 tx=0x7f543c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec679aab5a2701f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:08.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5437fff640 1 -- 172.21.15.92:0/671229753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c01d070 con 0x55fef8fb02f0 2024-09-11T22:14:08.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5437fff640 1 -- 172.21.15.92:0/671229753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f543c00ce30 con 0x55fef8fb02f0 2024-09-11T22:14:08.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.668+0000 7f5437fff640 1 -- 172.21.15.92:0/671229753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c00e410 con 0x55fef8fb02f0 2024-09-11T22:14:08.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 -- 172.21.15.92:0/671229753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fb02f0 msgr2=0x55fef8fd4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fb02f0 0x55fef8fd4b80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f543c0058c0 tx=0x7f543c00cbe0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 -- 172.21.15.92:0/671229753 shutdown_connections 2024-09-11T22:14:08.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8fb02f0 0x55fef8fd4b80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 --2- 172.21.15.92:0/671229753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8faf9d0 0x55fef8fc7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 -- 172.21.15.92:0/671229753 >> 172.21.15.92:0/671229753 conn(0x55fef8fabf60 msgr2=0x55fef8fca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 -- 172.21.15.92:0/671229753 shutdown_connections 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 -- 172.21.15.92:0/671229753 wait complete. 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 -- start start 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.669+0000 7f5456a61740 1 Processor -- start 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fb4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fcc720 con 0x55fef8faf9d0 2024-09-11T22:14:08.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fcc050 con 0x55fef8fbabb0 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fef8fccdf0 con 0x55fef8fb02f0 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 10 client.0 _ll_register_callbacks cb 0x7f5444022020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fb4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fb4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38638/0 (socket says 172.21.15.92:38638) 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 -- 172.21.15.92:0/3966381544 learned_addr learned my addr 172.21.15.92:0/3966381544 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5451c23640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452c25640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 -- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 msgr2=0x55fef8fb4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fb4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 -- 172.21.15.92:0/3966381544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 msgr2=0x55fef8fd3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 -- 172.21.15.92:0/3966381544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef8fd40b0 con 0x55fef8faf9d0 2024-09-11T22:14:08.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452c25640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5451c23640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fb4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5452424640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fb4130 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f54480b88e0 tx=0x7f5448003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d7636c0625bb99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5435ffb640 1 -- 172.21.15.92:0/3966381544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448049040 con 0x55fef8faf9d0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5435ffb640 1 -- 172.21.15.92:0/3966381544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f54480ed410 con 0x55fef8faf9d0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f543c00ce30 con 0x55fef8faf9d0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.670+0000 7f5435ffb640 1 -- 172.21.15.92:0/3966381544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54480c02c0 con 0x55fef8faf9d0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 10 client.15231 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fef8fd5320 con 0x55fef8faf9d0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5435ffb640 1 -- 172.21.15.92:0/3966381544 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f543c00ce30 con 0x55fef8faf9d0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5435ffb640 1 -- 172.21.15.92:0/3966381544 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f544803bc60 con 0x55fef8faf9d0 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 10 client.15231 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 10 client.15231 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T22:14:08.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 20 client.15231 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef903a080 0x55fef9038d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 4 client.15231 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 --> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] -- command(tid 0: {"prefix": "status"}) -- 0x55fef8f1db10 con 0x55fef903a080 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.671+0000 7f5451c23640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef903a080 0x55fef9038d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5451c23640 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef903a080 0x55fef9038d90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=795fa29105d6f5a8 server_cookie=361a70f92b750f6b in_seq=0 out_seq=0 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5435ffb640 10 client.15231 ms_handle_connect on v2:172.21.15.153:6834/1638990868 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5435ffb640 1 -- 172.21.15.92:0/3966381544 <== mds.4 v2:172.21.15.153:6834/1638990868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fef8f1db10 con 0x55fef903a080 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5435ffb640 10 client.15231 handle_command_reply: tid=0 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 client.15231 shutdown 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 msgr2=0x55fef8fb4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fb4130 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f54480b88e0 tx=0x7f5448003f20 comp rx=0 tx=0).stop 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 shutdown_connections 2024-09-11T22:14:08.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:6834/1638990868,v1:172.21.15.153:6835/1638990868] conn(0x55fef903a080 0x55fef9038d90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5435ffb640 0 client.15231 ms_handle_reset on v2:172.21.15.153:6834/1638990868 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fef8fbabb0 0x55fef8fd3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fef8fb02f0 0x55fef8fb4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 --2- 172.21.15.92:0/3966381544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fef8faf9d0 0x55fef8fb4130 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 >> 172.21.15.92:0/3966381544 conn(0x55fef8fabf60 msgr2=0x55fef8fb1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 shutdown_connections 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 1 -- 172.21.15.92:0/3966381544 wait complete. 2024-09-11T22:14:08.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.672+0000 7f5456a61740 20 client.15231 trim_cache size 0 max 16384 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24427, 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 89.528941391999993, 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 151.85721584300001 2024-09-11T22:14:08.689 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:14:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f54380a2c10 msgr2=0x7f54380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f54380a2c10 0x7f54380a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5444023000 tx=0x7f5444020000 comp rx=0 tx=0).stop 2024-09-11T22:14:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c100a80 msgr2=0x7f544c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c100a80 0x7f544c1991c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5448005940 tx=0x7f54480042f0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 shutdown_connections 2024-09-11T22:14:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f54380a2c10 0x7f54380a50d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f544c106a80 0x7f544c1a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c101420 0x7f544c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 --2- 172.21.15.92:0/4242475610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c100a80 0x7f544c1991c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 >> 172.21.15.92:0/4242475610 conn(0x7f544c0fc7a0 msgr2=0x7f544c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 shutdown_connections 2024-09-11T22:14:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:08.673+0000 7f5454688640 1 -- 172.21.15.92:0/4242475610 wait complete. 2024-09-11T22:14:08.992 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:08 smithi092 ceph-mon[31173]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:09.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:08 smithi153 ceph-mon[35045]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:08 smithi110 ceph-mon[34830]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:09.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:14:09.631 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:10.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 -- 172.21.15.92:0/3729351566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 msgr2=0x7fdad410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/3729351566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad410d930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009cd0 tx=0x7fdacc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 -- 172.21.15.92:0/3729351566 shutdown_connections 2024-09-11T22:14:10.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/3729351566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad410d930 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/3729351566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdad4100a60 0x7fdad4100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/3729351566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad4106a60 0x7fdad4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 -- 172.21.15.92:0/3729351566 >> 172.21.15.92:0/3729351566 conn(0x7fdad40fc780 msgr2=0x7fdad40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 -- 172.21.15.92:0/3729351566 shutdown_connections 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 -- 172.21.15.92:0/3729351566 wait complete. 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 Processor -- start 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.110+0000 7fdad8a8c640 1 -- start start 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad8a8c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdad4100a60 0x7fdad4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad8a8c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad8a8c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad4106a60 0x7fdad419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad8a8c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4110db0 con 0x7fdad4101400 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad8a8c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4110c30 con 0x7fdad4100a60 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad8a8c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4110f30 con 0x7fdad4106a60 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38666/0 (socket says 172.21.15.92:38666) 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 -- 172.21.15.92:0/632054227 learned_addr learned my addr 172.21.15.92:0/632054227 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad259c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdad4100a60 0x7fdad4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 -- 172.21.15.92:0/632054227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad4106a60 msgr2=0x7fdad419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad4106a60 0x7fdad419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad4106a60 0x7fdad419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 -- 172.21.15.92:0/632054227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdad4100a60 msgr2=0x7fdad4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdad4100a60 0x7fdad4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 -- 172.21.15.92:0/632054227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad419d440 con 0x7fdad4101400 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad259c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdad4100a60 0x7fdad4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad4106a60 0x7fdad419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacbfff640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad41989d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00b4a0 tx=0x7fdabc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacb7fe640 1 -- 172.21.15.92:0/632054227 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc014070 con 0x7fdad4101400 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.111+0000 7fdacb7fe640 1 -- 172.21.15.92:0/632054227 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc010ac0 con 0x7fdad4101400 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.112+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad419d730 con 0x7fdad4101400 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.112+0000 7fdacb7fe640 1 -- 172.21.15.92:0/632054227 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc017c30 con 0x7fdad4101400 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.112+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41a5010 con 0x7fdad4101400 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.114+0000 7fdacb7fe640 1 -- 172.21.15.92:0/632054227 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdabc017070 con 0x7fdad4101400 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.114+0000 7fdadae65740 1 Processor -- start 2024-09-11T22:14:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.114+0000 7fdadae65740 1 -- start start 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdadae65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdadae65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdadae65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdadae65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cfca720090 con 0x55cfca7c4bb0 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdadae65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfca6c7a70 con 0x55cfca7ba2f0 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdadae65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfca71c050 con 0x55cfca7b99d0 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacb7fe640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdac00a2c10 0x7fdac00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdac00a2c10 0x7fdac00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38672/0 (socket says 172.21.15.92:38672) 2024-09-11T22:14:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 -- 172.21.15.92:0/1630878059 learned_addr learned my addr 172.21.15.92:0/1630878059 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacb7fe640 1 -- 172.21.15.92:0/632054227 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdabc0ceba0 con 0x7fdad4101400 2024-09-11T22:14:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdad259c640 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 -- 172.21.15.92:0/1630878059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 msgr2=0x55cfca7b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 -- 172.21.15.92:0/1630878059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7ba2f0 msgr2=0x55cfca7c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 -- 172.21.15.92:0/1630878059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfca6d4130 con 0x55cfca7c4bb0 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdad259c640 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdacbfff640 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c6fa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00be60 tx=0x7fdabc06e110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25736460dfe35f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdac00a2c10 0x7fdac00a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0317f0 tx=0x7fdacc032fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.115+0000 7fdac8ff9640 1 -- 172.21.15.92:0/1630878059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc01e070 con 0x55cfca7c4bb0 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdac8ff9640 1 -- 172.21.15.92:0/1630878059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc09b050 con 0x55cfca7c4bb0 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdac8ff9640 1 -- 172.21.15.92:0/1630878059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0cc440 con 0x55cfca7c4bb0 2024-09-11T22:14:10.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 -- 172.21.15.92:0/1630878059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7c4bb0 msgr2=0x55cfca7c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c6fa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00be60 tx=0x7fdabc06e110 comp rx=0 tx=0).stop 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 -- 172.21.15.92:0/1630878059 shutdown_connections 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c6fa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 --2- 172.21.15.92:0/1630878059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 -- 172.21.15.92:0/1630878059 >> 172.21.15.92:0/1630878059 conn(0x55cfca7b5f60 msgr2=0x55cfca7b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 -- 172.21.15.92:0/1630878059 shutdown_connections 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.116+0000 7fdadae65740 1 -- 172.21.15.92:0/1630878059 wait complete. 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 -- start start 2024-09-11T22:14:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 Processor -- start 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7c7780 con 0x55cfca7ba2f0 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfca6daf20 con 0x55cfca7c4bb0 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7c7900 con 0x55cfca7b99d0 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdadae65740 10 client.0 _ll_register_callbacks cb 0x55cfca6a4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38688/0 (socket says 172.21.15.92:38688) 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdacbfff640 1 -- 172.21.15.92:0/3694304878 learned_addr learned my addr 172.21.15.92:0/3694304878 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.117+0000 7fdad259c640 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdacbfff640 1 -- 172.21.15.92:0/3694304878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 msgr2=0x55cfca7c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdacbfff640 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdacbfff640 1 -- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 msgr2=0x55cfca7c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdacbfff640 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdacbfff640 1 -- 172.21.15.92:0/3694304878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfca4912b0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdacbfff640 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c17a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdabc000c00 tx=0x7fdabc0604a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a5333a72fb2e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdab9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3694304878 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc01e050 con 0x55cfca7ba2f0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3694304878 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdabc0976c0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdabc09b050 con 0x55cfca7ba2f0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3694304878 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdabc010c70 con 0x55cfca7ba2f0 2024-09-11T22:14:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 10 client.15243 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cfca7d30a0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3694304878 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc09b050 con 0x55cfca7ba2f0 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3694304878 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fdabc095bc0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 10 client.15243 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 10 client.15243 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 20 client.15243 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca7d43f0 0x55cfca7d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 4 client.15243 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.118+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 --> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfca72d600 con 0x55cfca7d43f0 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.119+0000 7fdad259c640 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca7d43f0 0x55cfca7d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.119+0000 7fdad259c640 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca7d43f0 0x55cfca7d65e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d51f4ef6a08c20d server_cookie=5ee70c822a31cd2b in_seq=0 out_seq=0 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.119+0000 7fdab9ffb640 10 client.15243 ms_handle_connect on v2:172.21.15.110:6836/3571280995 2024-09-11T22:14:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3694304878 <== mds.3 v2:172.21.15.110:6836/3571280995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfca72d600 con 0x55cfca7d43f0 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdab9ffb640 10 client.15243 handle_command_reply: tid=0 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 client.15243 shutdown 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 msgr2=0x55cfca7c17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c17a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdabc000c00 tx=0x7fdabc0604a0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 shutdown_connections 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7c17a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca7d43f0 0x55cfca7d65e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 --2- 172.21.15.92:0/3694304878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdab9ffb640 0 client.15243 ms_handle_reset on v2:172.21.15.110:6836/3571280995 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 >> 172.21.15.92:0/3694304878 conn(0x55cfca7b5f60 msgr2=0x55cfca7bbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 shutdown_connections 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 1 -- 172.21.15.92:0/3694304878 wait complete. 2024-09-11T22:14:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.120+0000 7fdadae65740 20 client.15243 trim_cache size 0 max 16384 2024-09-11T22:14:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 Processor -- start 2024-09-11T22:14:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 -- start start 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7d4920 con 0x55cfca7ba2f0 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7d4ff0 con 0x55cfca7c4bb0 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdadae65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7d56c0 con 0x55cfca7b99d0 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38694/0 (socket says 172.21.15.92:38694) 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 -- 172.21.15.92:0/775363911 learned_addr learned my addr 172.21.15.92:0/775363911 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 -- 172.21.15.92:0/775363911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 msgr2=0x55cfca7d65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 -- 172.21.15.92:0/775363911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 msgr2=0x55cfca7d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7d4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdacbfff640 1 -- 172.21.15.92:0/775363911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfca7b5a00 con 0x55cfca7ba2f0 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.138+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdacbfff640 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d5840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cc6b0 tx=0x7fdabc0a3bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7019e0b6a57a137c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdabbfff640 1 -- 172.21.15.92:0/775363911 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc01e070 con 0x55cfca7ba2f0 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdabbfff640 1 -- 172.21.15.92:0/775363911 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc09b050 con 0x55cfca7ba2f0 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdabbfff640 1 -- 172.21.15.92:0/775363911 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc095bc0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 -- 172.21.15.92:0/775363911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 msgr2=0x55cfca7d5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d5840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cc6b0 tx=0x7fdabc0a3bf0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 -- 172.21.15.92:0/775363911 shutdown_connections 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca7d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d5840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 --2- 172.21.15.92:0/775363911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 -- 172.21.15.92:0/775363911 >> 172.21.15.92:0/775363911 conn(0x55cfca7b5f60 msgr2=0x55cfca7b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 -- 172.21.15.92:0/775363911 shutdown_connections 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.139+0000 7fdadae65740 1 -- 172.21.15.92:0/775363911 wait complete. 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 -- start start 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 Processor -- start 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca83d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7d6460 con 0x55cfca7ba2f0 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7d5d90 con 0x55cfca7c4bb0 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfca7d6b30 con 0x55cfca7b99d0 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdadae65740 10 client.0 _ll_register_callbacks cb 0x7fdabc017360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38710/0 (socket says 172.21.15.92:38710) 2024-09-11T22:14:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 -- 172.21.15.92:0/3173760775 learned_addr learned my addr 172.21.15.92:0/3173760775 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdad259c640 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdad2d9d640 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca83d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 -- 172.21.15.92:0/3173760775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 msgr2=0x55cfca7d7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7d7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 -- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 msgr2=0x55cfca83d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca83d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.140+0000 7fdacbfff640 1 -- 172.21.15.92:0/3173760775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfca83dde0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdacbfff640 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d84a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cef10 tx=0x7fdabc09b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7676c47f04e38b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdab9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3173760775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc01e070 con 0x55cfca7ba2f0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3173760775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdabc09a4f0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdabc09b050 con 0x55cfca7ba2f0 2024-09-11T22:14:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3173760775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdabc095dd0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 10 client.15249 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cfca83dde0 con 0x55cfca7ba2f0 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3173760775 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc09b050 con 0x55cfca7ba2f0 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3173760775 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fdabc084080 con 0x55cfca7ba2f0 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 10 client.15249 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 10 client.15249 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 20 client.15249 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca843510 0x55cfca842220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 4 client.15249 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.141+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 --> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] -- command(tid 0: {"prefix": "status"}) -- 0x55cfca727b10 con 0x55cfca843510 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdad259c640 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca843510 0x55cfca842220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdad259c640 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca843510 0x55cfca842220 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f54f11915edc5652 server_cookie=e5332de9f14dc0e1 in_seq=0 out_seq=0 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdab9ffb640 10 client.15249 ms_handle_connect on v2:172.21.15.110:6836/3571280995 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdab9ffb640 1 -- 172.21.15.92:0/3173760775 <== mds.3 v2:172.21.15.110:6836/3571280995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cfca727b10 con 0x55cfca843510 2024-09-11T22:14:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdab9ffb640 10 client.15249 handle_command_reply: tid=0 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 client.15249 shutdown 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 msgr2=0x55cfca7d84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d84a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cef10 tx=0x7fdabc09b7d0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 shutdown_connections 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfca7c4bb0 0x55cfca83d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfca7ba2f0 0x55cfca7d84a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.110:6836/3571280995,v1:172.21.15.110:6837/3571280995] conn(0x55cfca843510 0x55cfca842220 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 --2- 172.21.15.92:0/3173760775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfca7b99d0 0x55cfca7d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.142+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 >> 172.21.15.92:0/3173760775 conn(0x55cfca7b5f60 msgr2=0x55cfca7bb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdab9ffb640 0 client.15249 ms_handle_reset on v2:172.21.15.110:6836/3571280995 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 shutdown_connections 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdadae65740 1 -- 172.21.15.92:0/3173760775 wait complete. 2024-09-11T22:14:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdadae65740 20 client.15249 trim_cache size 0 max 16384 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24428, 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 92.009094680999993, 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:14:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:14:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:14:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 148.121028994 2024-09-11T22:14:10.157 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:14:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdac00a2c10 msgr2=0x7fdac00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdac00a2c10 0x7fdac00a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0317f0 tx=0x7fdacc032fd0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 msgr2=0x7fdad41989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad41989d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00b4a0 tx=0x7fdabc00b960 comp rx=0 tx=0).stop 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 shutdown_connections 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fdac00a2c10 0x7fdac00a50d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad4106a60 0x7fdad419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4101400 0x7fdad41989d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 --2- 172.21.15.92:0/632054227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdad4100a60 0x7fdad4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 >> 172.21.15.92:0/632054227 conn(0x7fdad40fc780 msgr2=0x7fdad406ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 shutdown_connections 2024-09-11T22:14:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:10.143+0000 7fdad8a8c640 1 -- 172.21.15.92:0/632054227 wait complete. 2024-09-11T22:14:10.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:10 smithi092 ceph-mon[31173]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:10.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:14:11.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:10 smithi153 ceph-mon[35045]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:11.175 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:10 smithi110 ceph-mon[34830]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:11.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 -- 172.21.15.92:0/1059965066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 msgr2=0x7f1550106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 --2- 172.21.15.92:0/1059965066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f1550106e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1540009930 tx=0x7f154002a420 comp rx=0 tx=0).stop 2024-09-11T22:14:11.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 -- 172.21.15.92:0/1059965066 shutdown_connections 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 --2- 172.21.15.92:0/1059965066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550101420 0x7f155010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 --2- 172.21.15.92:0/1059965066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1550100a80 0x7f1550100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 --2- 172.21.15.92:0/1059965066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f1550106e60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 -- 172.21.15.92:0/1059965066 >> 172.21.15.92:0/1059965066 conn(0x7f15500fc7a0 msgr2=0x7f15500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 -- 172.21.15.92:0/1059965066 shutdown_connections 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.679+0000 7f1555ecd640 1 -- 172.21.15.92:0/1059965066 wait complete. 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 Processor -- start 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 -- start start 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550100a80 0x7f1550199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1550101420 0x7f15501998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f15501a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1550110dd0 con 0x7f1550106a80 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1550110c50 con 0x7f1550101420 2024-09-11T22:14:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1555ecd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1550110f50 con 0x7f1550100a80 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f15501a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f15501a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38726/0 (socket says 172.21.15.92:38726) 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 -- 172.21.15.92:0/3600852753 learned_addr learned my addr 172.21.15.92:0/3600852753 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550100a80 0x7f1550199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1547fff640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1550101420 0x7f15501998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 -- 172.21.15.92:0/3600852753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550100a80 msgr2=0x7f1550199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550100a80 0x7f1550199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 -- 172.21.15.92:0/3600852753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1550101420 msgr2=0x7f15501998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1550101420 0x7f15501998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f154ffff640 1 -- 172.21.15.92:0/3600852753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15501a4940 con 0x7f1550106a80 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.680+0000 7f1547fff640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1550101420 0x7f15501998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.681+0000 7f154ffff640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f15501a4400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f154800b970 tx=0x7f154800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.681+0000 7f154d7fa640 1 -- 172.21.15.92:0/3600852753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548019070 con 0x7f1550106a80 2024-09-11T22:14:11.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.681+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15501a4b10 con 0x7f1550106a80 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.681+0000 7f154d7fa640 1 -- 172.21.15.92:0/3600852753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1548014440 con 0x7f1550106a80 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.681+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15501a5020 con 0x7f1550106a80 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.681+0000 7f154d7fa640 1 -- 172.21.15.92:0/3600852753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154801d5d0 con 0x7f1550106a80 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.683+0000 7f154d7fa640 1 -- 172.21.15.92:0/3600852753 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f154801c070 con 0x7f1550106a80 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.683+0000 7f15582a6740 1 Processor -- start 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.683+0000 7f15582a6740 1 -- start start 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f154d7fa640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f153c0a2c10 0x7f153c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f15582a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae289d0 0x55eeaae28db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f15582a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae33670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f15582a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae33bb0 0x55eeaae35fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f15582a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eeaad8f090 con 0x55eeaae292f0 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f15582a6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eeaad36a70 con 0x55eeaae33bb0 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f15582a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eeaad8b050 con 0x55eeaae289d0 2024-09-11T22:14:11.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f153c0a2c10 0x7f153c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f154d7fa640 1 -- 172.21.15.92:0/3600852753 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f15480ce990 con 0x7f1550106a80 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae33670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae33670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38742/0 (socket says 172.21.15.92:38742) 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 -- 172.21.15.92:0/2151474690 learned_addr learned my addr 172.21.15.92:0/2151474690 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f154f7fe640 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae289d0 0x55eeaae28db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f154ffff640 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae33bb0 0x55eeaae35fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 -- 172.21.15.92:0/2151474690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae289d0 msgr2=0x55eeaae28db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae289d0 0x55eeaae28db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 -- 172.21.15.92:0/2151474690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae33bb0 msgr2=0x55eeaae35fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae33bb0 0x55eeaae35fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 -- 172.21.15.92:0/2151474690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeaad43130 con 0x55eeaae292f0 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f153c0a2c10 0x7f153c0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f153800afd0 tx=0x7f153800a9b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:11.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1547fff640 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae33670 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1538014fd0 tx=0x7f1538019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca73fb515f24ba37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.684+0000 7f1545ffb640 1 -- 172.21.15.92:0/2151474690 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538025070 con 0x55eeaae292f0 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f1545ffb640 1 -- 172.21.15.92:0/2151474690 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153801c070 con 0x55eeaae292f0 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f1545ffb640 1 -- 172.21.15.92:0/2151474690 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538020680 con 0x55eeaae292f0 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 -- 172.21.15.92:0/2151474690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 msgr2=0x55eeaae33670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae33670 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1538014fd0 tx=0x7f1538019000 comp rx=0 tx=0).stop 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 -- 172.21.15.92:0/2151474690 shutdown_connections 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae33bb0 0x55eeaae35fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae33670 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 --2- 172.21.15.92:0/2151474690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae289d0 0x55eeaae28db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 -- 172.21.15.92:0/2151474690 >> 172.21.15.92:0/2151474690 conn(0x55eeaae24f60 msgr2=0x55eeaae25340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 -- 172.21.15.92:0/2151474690 shutdown_connections 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.685+0000 7f15582a6740 1 -- 172.21.15.92:0/2151474690 wait complete. 2024-09-11T22:14:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 -- start start 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 Processor -- start 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae30730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae33bb0 0x55eeaae30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae36780 con 0x55eeaae292f0 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eeaad49f20 con 0x55eeaae289d0 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae36900 con 0x55eeaae33bb0 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f15582a6740 10 client.0 _ll_register_callbacks cb 0x55eeaad13030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae30730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae30730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38746/0 (socket says 172.21.15.92:38746) 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 -- 172.21.15.92:0/3909920455 learned_addr learned my addr 172.21.15.92:0/3909920455 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f154ffff640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae33bb0 0x55eeaae30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 -- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae33bb0 msgr2=0x55eeaae30c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae33bb0 0x55eeaae30c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 -- 172.21.15.92:0/3909920455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 msgr2=0x55eeaae333e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae333e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f1547fff640 1 -- 172.21.15.92:0/3909920455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeaab002b0 con 0x55eeaae292f0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.686+0000 7f154ffff640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae33bb0 0x55eeaae30c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f1547fff640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae30730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1538016fd0 tx=0x7f1538018930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fba1a7ae75b07f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f152effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f152effd640 1 -- 172.21.15.92:0/3909920455 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538025040 con 0x55eeaae292f0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f153801c070 con 0x55eeaae292f0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f152effd640 1 -- 172.21.15.92:0/3909920455 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f153802b430 con 0x55eeaae292f0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f152effd640 1 -- 172.21.15.92:0/3909920455 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1538038050 con 0x55eeaae292f0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 10 client.15261 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eeaae42030 con 0x55eeaae292f0 2024-09-11T22:14:11.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f152effd640 1 -- 172.21.15.92:0/3909920455 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153801c070 con 0x55eeaae292f0 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f152effd640 1 -- 172.21.15.92:0/3909920455 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f1538015040 con 0x55eeaae292f0 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 10 client.15261 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 20 client.15261 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaae44280 0x55eeaae456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 4 client.15261 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.687+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 --> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeaad9c600 con 0x55eeaae44280 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.688+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaae44280 0x55eeaae456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.688+0000 7f152effd640 10 client.15261 ms_handle_connect on v2:172.21.15.153:6836/1434479445 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.688+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaae44280 0x55eeaae456f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6345aff7db32321d server_cookie=1323c8a0d802b6d3 in_seq=0 out_seq=0 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f152effd640 1 -- 172.21.15.92:0/3909920455 <== mds.2 v2:172.21.15.153:6836/1434479445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eeaad9c600 con 0x55eeaae44280 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f152effd640 10 client.15261 handle_command_reply: tid=0 2024-09-11T22:14:11.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 client.15261 shutdown 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 msgr2=0x55eeaae30730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae30730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1538016fd0 tx=0x7f1538018930 comp rx=0 tx=0).stop 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 shutdown_connections 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaae44280 0x55eeaae456f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae33bb0 0x55eeaae30c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae292f0 0x55eeaae30730 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 --2- 172.21.15.92:0/3909920455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae333e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f152effd640 0 client.15261 ms_handle_reset on v2:172.21.15.153:6836/1434479445 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 >> 172.21.15.92:0/3909920455 conn(0x55eeaae24f60 msgr2=0x55eeaae2ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 shutdown_connections 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 1 -- 172.21.15.92:0/3909920455 wait complete. 2024-09-11T22:14:11.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.689+0000 7f15582a6740 20 client.15261 trim_cache size 0 max 16384 2024-09-11T22:14:11.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 Processor -- start 2024-09-11T22:14:11.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 -- start start 2024-09-11T22:14:11.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae44280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae43bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae43a30 con 0x55eeaae33bb0 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae44100 con 0x55eeaae289d0 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f15582a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae447d0 con 0x55eeaae292f0 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f154ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae43bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f154ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae43bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38758/0 (socket says 172.21.15.92:38758) 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f154ffff640 1 -- 172.21.15.92:0/2340675676 learned_addr learned my addr 172.21.15.92:0/2340675676 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.707+0000 7f154f7fe640 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae44280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f1547fff640 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f154ffff640 1 -- 172.21.15.92:0/2340675676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 msgr2=0x55eeaae456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f154ffff640 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f154ffff640 1 -- 172.21.15.92:0/2340675676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 msgr2=0x55eeaae44280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f154ffff640 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae44280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f154ffff640 1 -- 172.21.15.92:0/2340675676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeaae29910 con 0x55eeaae33bb0 2024-09-11T22:14:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f1547fff640 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f154ffff640 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae43bb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f154809abf0 tx=0x7f15480bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73e004c7216a8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f15457fa640 1 -- 172.21.15.92:0/2340675676 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154809d070 con 0x55eeaae33bb0 2024-09-11T22:14:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f15457fa640 1 -- 172.21.15.92:0/2340675676 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f154809aef0 con 0x55eeaae33bb0 2024-09-11T22:14:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.708+0000 7f15457fa640 1 -- 172.21.15.92:0/2340675676 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480145f0 con 0x55eeaae33bb0 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 -- 172.21.15.92:0/2340675676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 msgr2=0x55eeaae43bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae43bb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f154809abf0 tx=0x7f15480bb040 comp rx=0 tx=0).stop 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 -- 172.21.15.92:0/2340675676 shutdown_connections 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae43bb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 --2- 172.21.15.92:0/2340675676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae44280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 -- 172.21.15.92:0/2340675676 >> 172.21.15.92:0/2340675676 conn(0x55eeaae24f60 msgr2=0x55eeaae2be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 -- 172.21.15.92:0/2340675676 shutdown_connections 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 -- 172.21.15.92:0/2340675676 wait complete. 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.709+0000 7f15582a6740 1 -- start start 2024-09-11T22:14:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 1 Processor -- start 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae2d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae45570 con 0x55eeaae33bb0 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae44ea0 con 0x55eeaae289d0 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae45c40 con 0x55eeaae292f0 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f15582a6740 10 client.0 _ll_register_callbacks cb 0x7f1538023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae2d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae2d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38768/0 (socket says 172.21.15.92:38768) 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 -- 172.21.15.92:0/3880173440 learned_addr learned my addr 172.21.15.92:0/3880173440 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f1547fff640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 -- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 msgr2=0x55eeaae2cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae2cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 -- 172.21.15.92:0/3880173440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 msgr2=0x55eeaae2caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae2caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 -- 172.21.15.92:0/3880173440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeaae4cb50 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f154ffff640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae2d520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f154809ae80 tx=0x7f1548004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfbd8bab6b7ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.710+0000 7f152effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f152effd640 1 -- 172.21.15.92:0/3880173440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154809d070 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f152effd640 1 -- 172.21.15.92:0/3880173440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1548094da0 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154809aef0 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f152effd640 1 -- 172.21.15.92:0/3880173440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f154809f1f0 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 10 client.15267 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eeaae4ddc0 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f152effd640 1 -- 172.21.15.92:0/3880173440 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f154809aef0 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f152effd640 1 -- 172.21.15.92:0/3880173440 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f154809a3d0 con 0x55eeaae33bb0 2024-09-11T22:14:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 10 client.15267 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 10 client.15267 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 20 client.15267 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaaeb2610 0x55eeaaeb1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 4 client.15267 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.711+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 --> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] -- command(tid 0: {"prefix": "status"}) -- 0x55eeaad96b10 con 0x55eeaaeb2610 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaaeb2610 0x55eeaaeb1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f154f7fe640 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaaeb2610 0x55eeaaeb1320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33e4eebfa659ba62 server_cookie=ce0a60595e510a35 in_seq=0 out_seq=0 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f152effd640 10 client.15267 ms_handle_connect on v2:172.21.15.153:6836/1434479445 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f152effd640 1 -- 172.21.15.92:0/3880173440 <== mds.2 v2:172.21.15.153:6836/1434479445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eeaad96b10 con 0x55eeaaeb2610 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f152effd640 10 client.15267 handle_command_reply: tid=0 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 client.15267 shutdown 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 msgr2=0x55eeaae2d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae2d520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f154809ae80 tx=0x7f1548004a40 comp rx=0 tx=0).stop 2024-09-11T22:14:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 shutdown_connections 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:6836/1434479445,v1:172.21.15.153:6837/1434479445] conn(0x55eeaaeb2610 0x55eeaaeb1320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eeaae33bb0 0x55eeaae2d520 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeaae292f0 0x55eeaae2cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f152effd640 0 client.15267 ms_handle_reset on v2:172.21.15.153:6836/1434479445 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 --2- 172.21.15.92:0/3880173440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eeaae289d0 0x55eeaae2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.712+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 >> 172.21.15.92:0/3880173440 conn(0x55eeaae24f60 msgr2=0x55eeaae2bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 shutdown_connections 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f15582a6740 1 -- 172.21.15.92:0/3880173440 wait complete. 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f15582a6740 20 client.15267 trim_cache size 0 max 16384 2024-09-11T22:14:11.719 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24433, 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 94.589375176000004, 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 152.32739301500001 2024-09-11T22:14:11.720 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f153c0a2c10 msgr2=0x7f153c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f153c0a2c10 0x7f153c0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f153800afd0 tx=0x7f153800a9b0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 msgr2=0x7f15501a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f15501a4400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f154800b970 tx=0x7f154800be30 comp rx=0 tx=0).stop 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 shutdown_connections 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f153c0a2c10 0x7f153c0a50d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1550106a80 0x7f15501a4400 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1550101420 0x7f15501998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 --2- 172.21.15.92:0/3600852753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1550100a80 0x7f1550199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:11.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 >> 172.21.15.92:0/3600852753 conn(0x7f15500fc7a0 msgr2=0x7f15500fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:11.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 shutdown_connections 2024-09-11T22:14:11.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:11.713+0000 7f1555ecd640 1 -- 172.21.15.92:0/3600852753 wait complete. 2024-09-11T22:14:12.418 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:12 smithi092 ceph-mon[31173]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:12.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:14:12.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:12 smithi110 ceph-mon[34830]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:12.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:12 smithi153 ceph-mon[35045]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:12.666 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:13.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 -- 172.21.15.92:0/3301633422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281082a0 msgr2=0x7f802810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 --2- 172.21.15.92:0/3301633422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281082a0 0x7f802810ec40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8020009cd0 tx=0x7f802002a910 comp rx=0 tx=0).stop 2024-09-11T22:14:13.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 -- 172.21.15.92:0/3301633422 shutdown_connections 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 --2- 172.21.15.92:0/3301633422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281082a0 0x7f802810ec40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 --2- 172.21.15.92:0/3301633422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80281036e0 0x7f8028107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 --2- 172.21.15.92:0/3301633422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8028102d30 0x7f8028103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 -- 172.21.15.92:0/3301633422 >> 172.21.15.92:0/3301633422 conn(0x7f80280fc7d0 msgr2=0x7f80280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 -- 172.21.15.92:0/3301633422 shutdown_connections 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 -- 172.21.15.92:0/3301633422 wait complete. 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 Processor -- start 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.132+0000 7f802dc80640 1 -- start start 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f802dc80640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8028102d30 0x7f80281984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f802dc80640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281036e0 0x7f8028198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f802dc80640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80281082a0 0x7f802819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f802dc80640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8028111f90 con 0x7f80281036e0 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f802dc80640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8028111e10 con 0x7f80281082a0 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f802dc80640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8028112110 con 0x7f8028102d30 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281036e0 0x7f8028198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281036e0 0x7f8028198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38792/0 (socket says 172.21.15.92:38792) 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 -- 172.21.15.92:0/715936988 learned_addr learned my addr 172.21.15.92:0/715936988 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f80277fe640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8028102d30 0x7f80281984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f8027fff640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80281082a0 0x7f802819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 -- 172.21.15.92:0/715936988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8028102d30 msgr2=0x7f80281984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8028102d30 0x7f80281984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 -- 172.21.15.92:0/715936988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80281082a0 msgr2=0x7f802819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80281082a0 0x7f802819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 -- 172.21.15.92:0/715936988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802819d470 con 0x7f80281036e0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f801ffff640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281036e0 0x7f8028198a00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8010006fd0 tx=0x7f801000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f80257fa640 1 -- 172.21.15.92:0/715936988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010011830 con 0x7f80281036e0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f80257fa640 1 -- 172.21.15.92:0/715936988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8010016070 con 0x7f80281036e0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802819d760 con 0x7f80281036e0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.133+0000 7f80257fa640 1 -- 172.21.15.92:0/715936988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010010df0 con 0x7f80281036e0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.134+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a5040 con 0x7f80281036e0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f80257fa640 1 -- 172.21.15.92:0/715936988 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f80100119d0 con 0x7f80281036e0 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 Processor -- start 2024-09-11T22:14:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 -- start start 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e1bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e26670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f80257fa640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f80140a2c10 0x7f80140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e28fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3d82090 con 0x55a4e3e26bb0 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3d29a70 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8030059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3d7e050 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f8027fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e28fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f80277fe640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f80140a2c10 0x7f80140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.136+0000 7f80257fa640 1 -- 172.21.15.92:0/715936988 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f80100cea70 con 0x7f80281036e0 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f8027fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e28fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38794/0 (socket says 172.21.15.92:38794) 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f8027fff640 1 -- 172.21.15.92:0/1262786667 learned_addr learned my addr 172.21.15.92:0/1262786667 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e26670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f8027fff640 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e1bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801ffff640 1 -- 172.21.15.92:0/1262786667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 msgr2=0x55a4e3e1bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801ffff640 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e1bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801ffff640 1 -- 172.21.15.92:0/1262786667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 msgr2=0x55a4e3e28fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801ffff640 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e28fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801ffff640 1 -- 172.21.15.92:0/1262786667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e3d36130 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f8027fff640 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e28fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f80277fe640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f80140a2c10 0x7f80140a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8018005d00 tx=0x7f801800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801ffff640 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e26670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80100cc1d0 tx=0x7f80100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=418750332f2a1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801dffb640 1 -- 172.21.15.92:0/1262786667 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801009b620 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801dffb640 1 -- 172.21.15.92:0/1262786667 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801009af20 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.137+0000 7f801dffb640 1 -- 172.21.15.92:0/1262786667 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010097590 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 -- 172.21.15.92:0/1262786667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1c2f0 msgr2=0x55a4e3e26670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e26670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80100cc1d0 tx=0x7f80100046c0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 -- 172.21.15.92:0/1262786667 shutdown_connections 2024-09-11T22:14:13.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e28fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e26670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 --2- 172.21.15.92:0/1262786667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e1bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 -- 172.21.15.92:0/1262786667 >> 172.21.15.92:0/1262786667 conn(0x55a4e3e17f60 msgr2=0x55a4e3e18340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 -- 172.21.15.92:0/1262786667 shutdown_connections 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 -- 172.21.15.92:0/1262786667 wait complete. 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.138+0000 7f8030059740 1 -- start start 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 1 Processor -- start 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e26480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e23d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e29780 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3d3cf20 con 0x55a4e3e26bb0 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e29900 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e26480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8030059740 10 client.0 _ll_register_callbacks cb 0x55a4e3d06030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e26480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38804/0 (socket says 172.21.15.92:38804) 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 -- 172.21.15.92:0/1200997300 learned_addr learned my addr 172.21.15.92:0/1200997300 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f801ffff640 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f8027fff640 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e23d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 -- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 msgr2=0x55a4e3e237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 -- 172.21.15.92:0/1200997300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 msgr2=0x55a4e3e23d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e23d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.139+0000 7f80277fe640 1 -- 172.21.15.92:0/1200997300 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e3af32b0 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f80277fe640 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e26480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f80180116c0 tx=0x7f8018004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70d776f0e05c02a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:13.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8006ffd640 1 -- 172.21.15.92:0/1200997300 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801801e8f0 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8006ffd640 1 -- 172.21.15.92:0/1200997300 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f801801ea90 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801009af20 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8006ffd640 1 -- 172.21.15.92:0/1200997300 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f801802f050 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 10 client.15279 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4e3e35130 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8006ffd640 1 -- 172.21.15.92:0/1200997300 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801009af20 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8006ffd640 1 -- 172.21.15.92:0/1200997300 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f801801d3b0 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 20 client.15279 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:13.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3e37770 0x55a4e3e38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 4 client.15279 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.140+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 --> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4e3d8f600 con 0x55a4e3e37770 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.141+0000 7f801ffff640 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3e37770 0x55a4e3e38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.141+0000 7f8006ffd640 10 client.15279 ms_handle_connect on v2:172.21.15.153:6838/229095273 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.141+0000 7f801ffff640 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3e37770 0x55a4e3e38be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7baf77528af200f server_cookie=648a6a466466399b in_seq=0 out_seq=0 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8006ffd640 1 -- 172.21.15.92:0/1200997300 <== mds.1 v2:172.21.15.153:6838/229095273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4e3d8f600 con 0x55a4e3e37770 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8006ffd640 10 client.15279 handle_command_reply: tid=0 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 client.15279 shutdown 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1b9d0 msgr2=0x55a4e3e26480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e26480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f80180116c0 tx=0x7f8018004730 comp rx=0 tx=0).stop 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 shutdown_connections 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e23d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3e37770 0x55a4e3e38be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 --2- 172.21.15.92:0/1200997300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e26480 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 >> 172.21.15.92:0/1200997300 conn(0x55a4e3e17f60 msgr2=0x55a4e3e1dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8006ffd640 0 client.15279 ms_handle_reset on v2:172.21.15.153:6838/229095273 2024-09-11T22:14:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 shutdown_connections 2024-09-11T22:14:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.142+0000 7f8030059740 1 -- 172.21.15.92:0/1200997300 wait complete. 2024-09-11T22:14:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.143+0000 7f8030059740 20 client.15279 trim_cache size 0 max 16384 2024-09-11T22:14:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 Processor -- start 2024-09-11T22:14:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 -- start start 2024-09-11T22:14:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e37e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e38510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e36f20 con 0x55a4e3e26bb0 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e375f0 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8030059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e37cc0 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e370a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38810/0 (socket says 172.21.15.92:38810) 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 -- 172.21.15.92:0/714686534 learned_addr learned my addr 172.21.15.92:0/714686534 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f80277fe640 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e37e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f801ffff640 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e38510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 -- 172.21.15.92:0/714686534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 msgr2=0x55a4e3e38510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e38510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 -- 172.21.15.92:0/714686534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1b9d0 msgr2=0x55a4e3e37e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e37e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.160+0000 7f8027fff640 1 -- 172.21.15.92:0/714686534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e3e1c120 con 0x55a4e3e26bb0 2024-09-11T22:14:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8027fff640 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e370a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f80200317e0 tx=0x7f802002a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0d78b64cad665a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f801d7fa640 1 -- 172.21.15.92:0/714686534 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020031c50 con 0x55a4e3e26bb0 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f801d7fa640 1 -- 172.21.15.92:0/714686534 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8020036040 con 0x55a4e3e26bb0 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f801d7fa640 1 -- 172.21.15.92:0/714686534 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020004780 con 0x55a4e3e26bb0 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 -- 172.21.15.92:0/714686534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 msgr2=0x55a4e3e370a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e370a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f80200317e0 tx=0x7f802002a910 comp rx=0 tx=0).stop 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 -- 172.21.15.92:0/714686534 shutdown_connections 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e370a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e38510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 --2- 172.21.15.92:0/714686534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e37e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 -- 172.21.15.92:0/714686534 >> 172.21.15.92:0/714686534 conn(0x55a4e3e17f60 msgr2=0x55a4e3e220a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.161+0000 7f8030059740 1 -- 172.21.15.92:0/714686534 shutdown_connections 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 -- 172.21.15.92:0/714686534 wait complete. 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 -- start start 2024-09-11T22:14:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 Processor -- start 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e3b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e3bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e38a60 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e38390 con 0x55a4e3e26bb0 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4e3e39130 con 0x55a4e3e1b9d0 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8030059740 10 client.0 _ll_register_callbacks cb 0x7f801800b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f801ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e3b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f801ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e3b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38820/0 (socket says 172.21.15.92:38820) 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f801ffff640 1 -- 172.21.15.92:0/505664440 learned_addr learned my addr 172.21.15.92:0/505664440 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.162+0000 7f8027fff640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e3bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f80277fe640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f801ffff640 1 -- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 msgr2=0x55a4e3e416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f801ffff640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f801ffff640 1 -- 172.21.15.92:0/505664440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 msgr2=0x55a4e3e3bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f801ffff640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e3bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f801ffff640 1 -- 172.21.15.92:0/505664440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e3e3c7e0 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8027fff640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e3bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f80277fe640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f801ffff640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e3b9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f80100cc580 tx=0x7f80100047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5e4217a314e7ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8006ffd640 1 -- 172.21.15.92:0/505664440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100b8250 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8006ffd640 1 -- 172.21.15.92:0/505664440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8010099220 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8020036040 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8006ffd640 1 -- 172.21.15.92:0/505664440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f801009b620 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8030059740 10 client.15285 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4e3ea0f80 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8006ffd640 1 -- 172.21.15.92:0/505664440 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8020036040 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.163+0000 7f8006ffd640 1 -- 172.21.15.92:0/505664440 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f8010097590 con 0x55a4e3e1c2f0 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f8030059740 10 client.15285 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f8030059740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f8030059740 20 client.15285 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f8030059740 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3ea5740 0x55a4e3ea4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f8030059740 4 client.15285 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 --> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] -- command(tid 0: {"prefix": "status"}) -- 0x55a4e3d89b10 con 0x55a4e3ea5740 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f80277fe640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3ea5740 0x55a4e3ea4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f8006ffd640 10 client.15285 ms_handle_connect on v2:172.21.15.153:6838/229095273 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.164+0000 7f80277fe640 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3ea5740 0x55a4e3ea4450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d51743ae8068c40 server_cookie=ee8862ccbdfefb6c in_seq=0 out_seq=0 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8006ffd640 1 -- 172.21.15.92:0/505664440 <== mds.1 v2:172.21.15.153:6838/229095273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a4e3d89b10 con 0x55a4e3ea5740 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8006ffd640 10 client.15285 handle_command_reply: tid=0 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 client.15285 shutdown 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1c2f0 msgr2=0x55a4e3e3b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e3b9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f80100cc580 tx=0x7f80100047d0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 shutdown_connections 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4e3e26bb0 0x55a4e3e3bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4e3e1c2f0 0x55a4e3e3b9c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:6838/229095273,v1:172.21.15.153:6839/229095273] conn(0x55a4e3ea5740 0x55a4e3ea4450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 --2- 172.21.15.92:0/505664440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4e3e1b9d0 0x55a4e3e416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 >> 172.21.15.92:0/505664440 conn(0x55a4e3e17f60 msgr2=0x55a4e3e228c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8006ffd640 0 client.15285 ms_handle_reset on v2:172.21.15.153:6838/229095273 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 shutdown_connections 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 1 -- 172.21.15.92:0/505664440 wait complete. 2024-09-11T22:14:13.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.165+0000 7f8030059740 20 client.15285 trim_cache size 0 max 16384 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24451, 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 97.047301497999996, 2024-09-11T22:14:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:14:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:14:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:14:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 148.641068773 2024-09-11T22:14:13.174 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f80140a2c10 msgr2=0x7f80140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f80140a2c10 0x7f80140a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8018005d00 tx=0x7f801800b410 comp rx=0 tx=0).stop 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281036e0 msgr2=0x7f8028198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281036e0 0x7f8028198a00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8010006fd0 tx=0x7f801000b8e0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 shutdown_connections 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f80140a2c10 0x7f80140a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80281082a0 0x7f802819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80281036e0 0x7f8028198a00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 --2- 172.21.15.92:0/715936988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8028102d30 0x7f80281984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 >> 172.21.15.92:0/715936988 conn(0x7f80280fc7d0 msgr2=0x7f8028105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 shutdown_connections 2024-09-11T22:14:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:13.166+0000 7f802dc80640 1 -- 172.21.15.92:0/715936988 wait complete. 2024-09-11T22:14:13.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:14:14.143 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:14.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 -- 172.21.15.92:0/1692489992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4106a80 msgr2=0x7fc9a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/1692489992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4106a80 0x7fc9a4106e60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009930 tx=0x7fc9a002a420 comp rx=0 tx=0).stop 2024-09-11T22:14:14.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 -- 172.21.15.92:0/1692489992 shutdown_connections 2024-09-11T22:14:14.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/1692489992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9a4101420 0x7fc9a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/1692489992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9a4100a80 0x7fc9a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/1692489992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4106a80 0x7fc9a4106e60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 -- 172.21.15.92:0/1692489992 >> 172.21.15.92:0/1692489992 conn(0x7fc9a40fc7a0 msgr2=0x7fc9a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 -- 172.21.15.92:0/1692489992 shutdown_connections 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.624+0000 7fc9ac64f640 1 -- 172.21.15.92:0/1692489992 wait complete. 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 Processor -- start 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 -- start start 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4100a80 0x7fc9a4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9a4101420 0x7fc9a4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9a4106a80 0x7fc9a419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4110d50 con 0x7fc9a4100a80 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4110bd0 con 0x7fc9a4106a80 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9ac64f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4110ed0 con 0x7fc9a4101420 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4100a80 0x7fc9a4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4100a80 0x7fc9a4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38846/0 (socket says 172.21.15.92:38846) 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.625+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2541326308 learned_addr learned my addr 172.21.15.92:0/2541326308 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9a4101420 0x7fc9a4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2541326308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9a4101420 msgr2=0x7fc9a4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9a4101420 0x7fc9a4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2541326308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9a4106a80 msgr2=0x7fc9a419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aabec640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9a4106a80 0x7fc9a419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9a4106a80 0x7fc9a419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2541326308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a419d2f0 con 0x7fc9a4100a80 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aabec640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9a4106a80 0x7fc9a419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4100a80 0x7fc9a4198340 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005940 tx=0x7fc9a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc99b7fe640 1 -- 172.21.15.92:0/2541326308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a003f030 con 0x7fc9a4100a80 2024-09-11T22:14:14.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a419d580 con 0x7fc9a4100a80 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc99b7fe640 1 -- 172.21.15.92:0/2541326308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a0003cb0 con 0x7fc9a4100a80 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc99b7fe640 1 -- 172.21.15.92:0/2541326308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0035070 con 0x7fc9a4100a80 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.626+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a41a4ec0 con 0x7fc9a4100a80 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.628+0000 7fc99b7fe640 1 -- 172.21.15.92:0/2541326308 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc9a0042030 con 0x7fc9a4100a80 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 Processor -- start 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 -- start start 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc99b7fe640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc9900a2c10 0x7fc9900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a4ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a4e2f0 0x560ac1a58670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a58bb0 0x560ac1a5afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560ac19b4090 con 0x560ac1a4d9d0 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560ac195ba70 con 0x560ac1a58bb0 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aea28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560ac19b0050 con 0x560ac1a4e2f0 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc99b7fe640 1 -- 172.21.15.92:0/2541326308 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc9a00efba0 con 0x7fc9a4100a80 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aabec640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc9900a2c10 0x7fc9900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a4ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a4ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38854/0 (socket says 172.21.15.92:38854) 2024-09-11T22:14:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2258032074 learned_addr learned my addr 172.21.15.92:0/2258032074 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.629+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a4e2f0 0x560ac1a58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aabec640 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a58bb0 0x560ac1a5afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2258032074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a4e2f0 msgr2=0x560ac1a58670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a4e2f0 0x560ac1a58670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2258032074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a58bb0 msgr2=0x560ac1a5afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a58bb0 0x560ac1a5afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2258032074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac1968130 con 0x560ac1a4d9d0 2024-09-11T22:14:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aabec640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc9900a2c10 0x7fc9900a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc99c013700 tx=0x7fc99c013b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a4ddb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009680 tx=0x7fc9a00bba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fa48399b520607 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc998ff9640 1 -- 172.21.15.92:0/2258032074 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0049040 con 0x560ac1a4d9d0 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc998ff9640 1 -- 172.21.15.92:0/2258032074 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a00bc640 con 0x560ac1a4d9d0 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.630+0000 7fc998ff9640 1 -- 172.21.15.92:0/2258032074 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a003b460 con 0x560ac1a4d9d0 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 -- 172.21.15.92:0/2258032074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 msgr2=0x560ac1a4ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a4ddb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009680 tx=0x7fc9a00bba80 comp rx=0 tx=0).stop 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 -- 172.21.15.92:0/2258032074 shutdown_connections 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a58bb0 0x560ac1a5afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a4e2f0 0x560ac1a58670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2258032074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a4ddb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 -- 172.21.15.92:0/2258032074 >> 172.21.15.92:0/2258032074 conn(0x560ac1a49f60 msgr2=0x560ac1a4a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 -- 172.21.15.92:0/2258032074 shutdown_connections 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 -- 172.21.15.92:0/2258032074 wait complete. 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 -- start start 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 Processor -- start 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.631+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a55810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a55d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a5b540 con 0x560ac1a4d9d0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560ac196ef20 con 0x560ac1a4e2f0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a5b6c0 con 0x560ac1a58bb0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 10 client.0 _ll_register_callbacks cb 0x560ac1938030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38860/0 (socket says 172.21.15.92:38860) 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2674553214 learned_addr learned my addr 172.21.15.92:0/2674553214 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a55810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aabec640 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a55d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2674553214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 msgr2=0x560ac1a55d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a55d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 msgr2=0x560ac1a55810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a55810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/2674553214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac17252b0 con 0x560ac1a4d9d0 2024-09-11T22:14:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a584c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00b8910 tx=0x7fc9a00b8860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcae343faf541a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc98dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc98dffb640 1 -- 172.21.15.92:0/2674553214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0049070 con 0x560ac1a4d9d0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc98dffb640 1 -- 172.21.15.92:0/2674553214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc9a003bc60 con 0x560ac1a4d9d0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a00bc640 con 0x560ac1a4d9d0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc98dffb640 1 -- 172.21.15.92:0/2674553214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9a00ed5b0 con 0x560ac1a4d9d0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 10 client.15297 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ac1a5b540 con 0x560ac1a4d9d0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.632+0000 7fc98dffb640 1 -- 172.21.15.92:0/2674553214 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a00bc640 con 0x560ac1a4d9d0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc98dffb640 1 -- 172.21.15.92:0/2674553214 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fc9a00c4720 con 0x560ac1a4d9d0 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9aea28740 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9aea28740 10 client.15297 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9aea28740 20 client.15297 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1a69270 0x560ac1a6a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9aea28740 4 client.15297 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 --> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ac19c1600 con 0x560ac1a69270 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1a69270 0x560ac1a6a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1a69270 0x560ac1a6a6e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d655fba2f4a29276 server_cookie=99fa8ed33985ed20 in_seq=0 out_seq=0 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.633+0000 7fc98dffb640 10 client.15297 ms_handle_connect on v2:172.21.15.110:6838/538708966 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc98dffb640 1 -- 172.21.15.92:0/2674553214 <== mds.0 v2:172.21.15.110:6838/538708966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ac19c1600 con 0x560ac1a69270 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc98dffb640 10 client.15297 handle_command_reply: tid=0 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 client.15297 shutdown 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 msgr2=0x560ac1a584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a584c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00b8910 tx=0x7fc9a00b8860 comp rx=0 tx=0).stop 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 shutdown_connections 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1a69270 0x560ac1a6a6e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a55d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a55810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc98dffb640 0 client.15297 ms_handle_reset on v2:172.21.15.110:6838/538708966 2024-09-11T22:14:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 --2- 172.21.15.92:0/2674553214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a584c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 >> 172.21.15.92:0/2674553214 conn(0x560ac1a49f60 msgr2=0x560ac1a4fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:14.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 shutdown_connections 2024-09-11T22:14:14.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.634+0000 7fc9aea28740 1 -- 172.21.15.92:0/2674553214 wait complete. 2024-09-11T22:14:14.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.635+0000 7fc9aea28740 20 client.15297 trim_cache size 0 max 16384 2024-09-11T22:14:14.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 Processor -- start 2024-09-11T22:14:14.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 -- start start 2024-09-11T22:14:14.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a52ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a68a20 con 0x560ac1a4d9d0 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a690f0 con 0x560ac1a4e2f0 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aea28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a697c0 con 0x560ac1a58bb0 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a52ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.652+0000 7fc9aa3eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a52ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38872/0 (socket says 172.21.15.92:38872) 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/3880386692 learned_addr learned my addr 172.21.15.92:0/3880386692 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aabec640 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/3880386692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 msgr2=0x560ac1a729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/3880386692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 msgr2=0x560ac1a52ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a52ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aa3eb640 1 -- 172.21.15.92:0/3880386692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac1a580e0 con 0x560ac1a4d9d0 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aabec640 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a52ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00092a0 tx=0x7fc9a002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d682aacadf2eac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc98ffff640 1 -- 172.21.15.92:0/3880386692 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00b81e0 con 0x560ac1a4d9d0 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc98ffff640 1 -- 172.21.15.92:0/3880386692 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a00753c0 con 0x560ac1a4d9d0 2024-09-11T22:14:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.653+0000 7fc98ffff640 1 -- 172.21.15.92:0/3880386692 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00c02f0 con 0x560ac1a4d9d0 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 -- 172.21.15.92:0/3880386692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 msgr2=0x560ac1a52ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a52ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00092a0 tx=0x7fc9a002a420 comp rx=0 tx=0).stop 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 -- 172.21.15.92:0/3880386692 shutdown_connections 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4e2f0 0x560ac1a52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 --2- 172.21.15.92:0/3880386692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4d9d0 0x560ac1a52ab0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 -- 172.21.15.92:0/3880386692 >> 172.21.15.92:0/3880386692 conn(0x560ac1a49f60 msgr2=0x560ac1a4a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 -- 172.21.15.92:0/3880386692 shutdown_connections 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 -- 172.21.15.92:0/3880386692 wait complete. 2024-09-11T22:14:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 -- start start 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.654+0000 7fc9aea28740 1 Processor -- start 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4d9d0 0x560ac1a73280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4e2f0 0x560ac1a6e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a6ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a6a560 con 0x560ac1a4e2f0 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a69e90 con 0x560ac1a4d9d0 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560ac1a6ac30 con 0x560ac1a58bb0 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 10 client.0 _ll_register_callbacks cb 0x7fc99400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4e2f0 0x560ac1a6e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4e2f0 0x560ac1a6e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38882/0 (socket says 172.21.15.92:38882) 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 -- 172.21.15.92:0/1251371585 learned_addr learned my addr 172.21.15.92:0/1251371585 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aabec640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a6ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4d9d0 0x560ac1a73280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 -- 172.21.15.92:0/1251371585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 msgr2=0x560ac1a6ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a6ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 -- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4d9d0 msgr2=0x560ac1a73280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4d9d0 0x560ac1a73280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 -- 172.21.15.92:0/1251371585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac1ad2980 con 0x560ac1a4e2f0 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4d9d0 0x560ac1a73280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aabec640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a6ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9a9bea640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4e2f0 0x560ac1a6e650 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x560ac1a550a0 tx=0x7fc99400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf1e44ecfc3cf93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc98dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc98dffb640 1 -- 172.21.15.92:0/1251371585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99401c470 con 0x560ac1a4e2f0 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc98dffb640 1 -- 172.21.15.92:0/1251371585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc99401c610 con 0x560ac1a4e2f0 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.655+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ac19bbb10 con 0x560ac1a4e2f0 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc98dffb640 1 -- 172.21.15.92:0/1251371585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc99402c050 con 0x560ac1a4e2f0 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 10 client.15303 fetch_fsmap learned FSMap version 27 2024-09-11T22:14:14.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ac1ad3b20 con 0x560ac1a4e2f0 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc98dffb640 1 -- 172.21.15.92:0/1251371585 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560ac19bbb10 con 0x560ac1a4e2f0 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc98dffb640 1 -- 172.21.15.92:0/1251371585 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fc994010b70 con 0x560ac1a4e2f0 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 10 client.15303 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 10 client.15303 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 20 client.15303 populate_metadata read hostname 'smithi092' 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1ad8740 0x560ac1ad7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 4 client.15303 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 --> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] -- command(tid 0: {"prefix": "status"}) -- 0x560ac19380e0 con 0x560ac1ad8740 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.656+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1ad8740 0x560ac1ad7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aa3eb640 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1ad8740 0x560ac1ad7450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb926efa27211181 server_cookie=8d656782c34a0fcd in_seq=0 out_seq=0 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc98dffb640 10 client.15303 ms_handle_connect on v2:172.21.15.110:6838/538708966 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc98dffb640 1 -- 172.21.15.92:0/1251371585 <== mds.0 v2:172.21.15.110:6838/538708966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560ac19380e0 con 0x560ac1ad8740 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc98dffb640 10 client.15303 handle_command_reply: tid=0 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 client.15303 shutdown 2024-09-11T22:14:14.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4e2f0 msgr2=0x560ac1a6e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4e2f0 0x560ac1a6e650 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x560ac1a550a0 tx=0x7fc99400cf00 comp rx=0 tx=0).stop 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 shutdown_connections 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:6838/538708966,v1:172.21.15.110:6839/538708966] conn(0x560ac1ad8740 0x560ac1ad7450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ac1a58bb0 0x560ac1a6ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ac1a4e2f0 0x560ac1a6e650 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 --2- 172.21.15.92:0/1251371585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ac1a4d9d0 0x560ac1a73280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 >> 172.21.15.92:0/1251371585 conn(0x560ac1a49f60 msgr2=0x560ac1a487e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc98dffb640 0 client.15303 ms_handle_reset on v2:172.21.15.110:6838/538708966 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 shutdown_connections 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 1 -- 172.21.15.92:0/1251371585 wait complete. 2024-09-11T22:14:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.657+0000 7fc9aea28740 20 client.15303 trim_cache size 0 max 16384 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "a6709e28-7089-11ef-bcea-c7b262605968", 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24455, 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 130.89254895900001, 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 27, 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 92, 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 141.70406992400001 2024-09-11T22:14:14.665 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:14:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc9900a2c10 msgr2=0x7fc9900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc9900a2c10 0x7fc9900a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc99c013700 tx=0x7fc99c013b00 comp rx=0 tx=0).stop 2024-09-11T22:14:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4100a80 msgr2=0x7fc9a4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4100a80 0x7fc9a4198340 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005940 tx=0x7fc9a00042f0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 shutdown_connections 2024-09-11T22:14:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fc9900a2c10 0x7fc9900a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9a4106a80 0x7fc9a419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9a4101420 0x7fc9a4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 --2- 172.21.15.92:0/2541326308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a4100a80 0x7fc9a4198340 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 >> 172.21.15.92:0/2541326308 conn(0x7fc9a40fc7a0 msgr2=0x7fc9a4104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 shutdown_connections 2024-09-11T22:14:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:14.658+0000 7fc9ac64f640 1 -- 172.21.15.92:0/2541326308 wait complete. 2024-09-11T22:14:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:14 smithi092 ceph-mon[31173]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:15.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:14 smithi153 ceph-mon[35045]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:14 smithi110 ceph-mon[34830]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:15.419 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T22:14:15.429 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T22:14:15.429 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T22:14:15.430 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T22:14:15.430 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-11T22:14:15.440 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-11T22:14:15.440 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-11T22:14:15.440 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-11T22:14:15.440 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T22:14:15.440 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:15.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:15.440 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-11T22:14:15.457 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-11T22:14:15.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:15.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T22:14:15.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:15.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-09-11T22:14:15.624 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-11T22:14:15.624 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-11T22:14:15.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T22:14:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 -- 172.21.15.92:0/2586227060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c106f10 msgr2=0x7f656c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 --2- 172.21.15.92:0/2586227060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c106f10 0x7f656c10ea90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6564009ca0 tx=0x7f656402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 -- 172.21.15.92:0/2586227060 shutdown_connections 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 --2- 172.21.15.92:0/2586227060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c106f10 0x7f656c10ea90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 --2- 172.21.15.92:0/2586227060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f656c102790 0x7f656c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 --2- 172.21.15.92:0/2586227060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656c101de0 0x7f656c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 -- 172.21.15.92:0/2586227060 >> 172.21.15.92:0/2586227060 conn(0x7f656c0fd8c0 msgr2=0x7f656c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:16.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 -- 172.21.15.92:0/2586227060 shutdown_connections 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.360+0000 7f657299d640 1 -- 172.21.15.92:0/2586227060 wait complete. 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.361+0000 7f657299d640 1 Processor -- start 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.361+0000 7f657299d640 1 -- start start 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.361+0000 7f657299d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c101de0 0x7f656c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f657299d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656c102790 0x7f656c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f657299d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f656c106f10 0x7f656c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f657299d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f656c111e40 con 0x7f656c101de0 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f657299d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f656c111cc0 con 0x7f656c106f10 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f657299d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f656c111fc0 con 0x7f656c102790 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c101de0 0x7f656c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c101de0 0x7f656c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47334/0 (socket says 172.21.15.92:47334) 2024-09-11T22:14:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 -- 172.21.15.92:0/3896949491 learned_addr learned my addr 172.21.15.92:0/3896949491 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f6570f3a640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f656c106f10 0x7f656c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656b7fe640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656c102790 0x7f656c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 -- 172.21.15.92:0/3896949491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656c102790 msgr2=0x7f656c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656c102790 0x7f656c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 -- 172.21.15.92:0/3896949491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f656c106f10 msgr2=0x7f656c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f656c106f10 0x7f656c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 -- 172.21.15.92:0/3896949491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c19d2d0 con 0x7f656c101de0 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656b7fe640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656c102790 0x7f656c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f656bfff640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c101de0 0x7f656c198320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f655c00b4a0 tx=0x7f655c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f65697fa640 1 -- 172.21.15.92:0/3896949491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c014070 con 0x7f656c101de0 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f65697fa640 1 -- 172.21.15.92:0/3896949491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f655c010ac0 con 0x7f656c101de0 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656c19d5c0 con 0x7f656c101de0 2024-09-11T22:14:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f65697fa640 1 -- 172.21.15.92:0/3896949491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c017c30 con 0x7f656c101de0 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.362+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656c1a4ea0 con 0x7f656c101de0 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.363+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656c1021c0 con 0x7f656c101de0 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.364+0000 7f65697fa640 1 -- 172.21.15.92:0/3896949491 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f655c017070 con 0x7f656c101de0 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.365+0000 7f65697fa640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65540a2c10 0x7f65540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.365+0000 7f656b7fe640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65540a2c10 0x7f65540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.365+0000 7f65697fa640 1 -- 172.21.15.92:0/3896949491 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f655c0ce810 con 0x7f656c101de0 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.366+0000 7f656b7fe640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65540a2c10 0x7f65540a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f656c1088e0 tx=0x7f6560005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:16.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.369+0000 7f65697fa640 1 -- 172.21.15.92:0/3896949491 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f655c09b3c0 con 0x7f656c101de0 2024-09-11T22:14:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.569+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f656c10a150 con 0x7f656c101de0 2024-09-11T22:14:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.569+0000 7f65697fa640 1 -- 172.21.15.92:0/3896949491 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f655c09acc0 con 0x7f656c101de0 2024-09-11T22:14:16.571 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65540a2c10 msgr2=0x7f65540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65540a2c10 0x7f65540a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f656c1088e0 tx=0x7f6560005d90 comp rx=0 tx=0).stop 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c101de0 msgr2=0x7f656c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c101de0 0x7f656c198320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f655c00b4a0 tx=0x7f655c00b960 comp rx=0 tx=0).stop 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 shutdown_connections 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f65540a2c10 0x7f65540a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f656c106f10 0x7f656c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656c102790 0x7f656c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 --2- 172.21.15.92:0/3896949491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f656c101de0 0x7f656c198320 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:16.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 >> 172.21.15.92:0/3896949491 conn(0x7f656c0fd8c0 msgr2=0x7f656c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:16.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 shutdown_connections 2024-09-11T22:14:16.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:16.572+0000 7f657299d640 1 -- 172.21.15.92:0/3896949491 wait complete. 2024-09-11T22:14:17.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:16 smithi092 ceph-mon[31173]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T22:14:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:16 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-11T22:14:17.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:16 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3896949491' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:14:17.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:16 smithi153 ceph-mon[35045]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T22:14:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:16 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-11T22:14:17.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:16 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3896949491' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:14:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:16 smithi110 ceph-mon[34830]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T22:14:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:16 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-11T22:14:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:16 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3896949491' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T22:14:17.389 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T22:14:17.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:17.390 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-09-11T22:14:17.406 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T22:14:17.406 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T22:14:17.406 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T22:14:17.406 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:14:17.406 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-09-11T22:14:17.407 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:14:17.407 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:14:17.407 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:14:17.407 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-09-11T22:14:17.407 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:14:17.407 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:14:17.407 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-09-11T22:14:17.408 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:14:17.408 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:14:17.408 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T22:14:17.408 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:14:17.408 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T22:14:17.408 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42022sec preferred_lft 42022sec 2024-09-11T22:14:17.409 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-11T22:14:17.409 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:14:17.409 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T22:14:17.409 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:14:17.409 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:14:17.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2024-09-11T22:14:17.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2024-09-11T22:14:17.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2024-09-11T22:14:17.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T22:14:17.410 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:14:17.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:14:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:14:17.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:17.549 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T22:14:17.573 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-11T22:14:17.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:17.574 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-09-11T22:14:17.631 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-09-11T22:14:17.631 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T22:14:17.631 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T22:14:17.631 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T22:14:17.631 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T22:14:17.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:14:17.632 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:14:17.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T22:14:17.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T22:14:17.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T22:14:17.632 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:14:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:14:17.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:14:17.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:17.760 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-11T22:14:17.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:17.775 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-09-11T22:14:17.832 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:14:17.832 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:14:17.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:14:17.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T22:14:17.832 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:14:17.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:14:17.930 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:17 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T22:14:17.930 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:17 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-11T22:14:17.930 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:17 smithi092 ceph-mon[31173]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-11T22:14:17.930 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:17 smithi092 ceph-mon[31173]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:14:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:14:17.952 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T22:14:17.952 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:14:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:14:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:17 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T22:14:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:17 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-11T22:14:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:17 smithi153 ceph-mon[35045]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-11T22:14:18.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:17 smithi153 ceph-mon[35045]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:14:18.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:14:18.068 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:14:18.068 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-11T22:14:18.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2024-09-11T22:14:18.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T22:14:18.068 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-11T22:14:18.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:14:18.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:14:18.145 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-11T22:14:18.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:14:18.146 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:18.174 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T22:14:18.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:14:18.175 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:17 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T22:14:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:17 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-11T22:14:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:17 smithi110 ceph-mon[34830]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-11T22:14:18.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:17 smithi110 ceph-mon[34830]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:14:18.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-11T22:14:18.297 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-11T22:14:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-11T22:14:18.354 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/205a3cf67e962caef53c7d9f16ee57f82186beeb3f77672d371a8524df0412e9/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b43dca6d6c930599632e7cbc1ffc10421bb5773b5661239002606220f430c440/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/0 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/149012ee1cfd020bbc35631c38da1b6c0dbef855b875ffafab34afc4f166aff0/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b9cd808df467a5a77d7a9b9f24fbe0421769b37ff3f487c6e39991de98e0c4c3/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e7d143339e84352e4cd39384d3592ea9059095e676b9f9939cf1da58ddfaf97a/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/72d2b71c6241585d2292458bd21d5f0d5a1804796666a88d95e8ef6740532c7e/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/439e56349dfaee14adfe3785a2e894297ce589a0e7a7ee20027fb9f0f054fee8/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/a53191da74c76582a71960335880c1792bc4f53c63be01d45dc8789f4502d30e/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e6b3cc7ab02e80c813880c96e15b6d432350447baa3c53da85fdc1e9928aed0e/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/dc98e095f781c132f81a0d6e06412b66681707091221fda195e4a3f25112c917/merged 2024-09-11T22:14:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-11T22:14:18.356 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:14:18.356 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:14:18.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:18.356 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-11T22:14:18.408 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-11T22:14:18.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/6d5615a6-07ce-4091-80b5-509f10920896 --client_fs=cephfs) 2024-09-11T22:14:18.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-11T22:14:18.475 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T22:14:18.531 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-11T22:14:18.532 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T22:14:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/205a3cf67e962caef53c7d9f16ee57f82186beeb3f77672d371a8524df0412e9/merged 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b43dca6d6c930599632e7cbc1ffc10421bb5773b5661239002606220f430c440/merged 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/0 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/149012ee1cfd020bbc35631c38da1b6c0dbef855b875ffafab34afc4f166aff0/merged 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b9cd808df467a5a77d7a9b9f24fbe0421769b37ff3f487c6e39991de98e0c4c3/merged 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e7d143339e84352e4cd39384d3592ea9059095e676b9f9939cf1da58ddfaf97a/merged 2024-09-11T22:14:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/72d2b71c6241585d2292458bd21d5f0d5a1804796666a88d95e8ef6740532c7e/merged 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/439e56349dfaee14adfe3785a2e894297ce589a0e7a7ee20027fb9f0f054fee8/merged 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/a53191da74c76582a71960335880c1792bc4f53c63be01d45dc8789f4502d30e/merged 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e6b3cc7ab02e80c813880c96e15b6d432350447baa3c53da85fdc1e9928aed0e/merged 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/dc98e095f781c132f81a0d6e06412b66681707091221fda195e4a3f25112c917/merged 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:14:18.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:18.535 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-11T22:14:18.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:2024-09-11T22:14:18.531+0000 7f472e40c480 -1 init, newargv = 0x561ac8b21c20 newargc=15 2024-09-11T22:14:18.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[86294]: starting ceph client 2024-09-11T22:14:18.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[86294]: starting fuse 2024-09-11T22:14:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:18 smithi092 ceph-mon[31173]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:14:19.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:18 smithi092 ceph-mon[31173]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:14:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:18 smithi153 ceph-mon[35045]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:14:19.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:18 smithi153 ceph-mon[35045]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:14:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:18 smithi110 ceph-mon[34830]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:14:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:18 smithi110 ceph-mon[34830]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:14:19.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-11T22:14:19.684 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T22:14:19.742 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-11T22:14:19.742 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-11T22:14:19.742 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-11T22:14:19.742 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-11T22:14:19.742 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-11T22:14:19.743 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/205a3cf67e962caef53c7d9f16ee57f82186beeb3f77672d371a8524df0412e9/merged 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b43dca6d6c930599632e7cbc1ffc10421bb5773b5661239002606220f430c440/merged 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/0 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/149012ee1cfd020bbc35631c38da1b6c0dbef855b875ffafab34afc4f166aff0/merged 2024-09-11T22:14:19.744 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b9cd808df467a5a77d7a9b9f24fbe0421769b37ff3f487c6e39991de98e0c4c3/merged 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e7d143339e84352e4cd39384d3592ea9059095e676b9f9939cf1da58ddfaf97a/merged 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/72d2b71c6241585d2292458bd21d5f0d5a1804796666a88d95e8ef6740532c7e/merged 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/439e56349dfaee14adfe3785a2e894297ce589a0e7a7ee20027fb9f0f054fee8/merged 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/a53191da74c76582a71960335880c1792bc4f53c63be01d45dc8789f4502d30e/merged 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/e6b3cc7ab02e80c813880c96e15b6d432350447baa3c53da85fdc1e9928aed0e/merged 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/dc98e095f781c132f81a0d6e06412b66681707091221fda195e4a3f25112c917/merged 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run.smithi092.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:19.746 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-11T22:14:19.800 INFO:teuthology.orchestra.run.smithi092.stdout:120 2024-09-11T22:14:19.801 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-11T22:14:19.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2024-09-11T22:14:19.801 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2024-09-11T22:14:19.801 DEBUG:teuthology.orchestra.run.smithi092:> import re 2024-09-11T22:14:19.801 DEBUG:teuthology.orchestra.run.smithi092:> import os 2024-09-11T22:14:19.801 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2024-09-11T22:14:19.801 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:14:19.801 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2024-09-11T22:14:19.802 DEBUG:teuthology.orchestra.run.smithi092:> return f 2024-09-11T22:14:19.803 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-11T22:14:19.803 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-11T22:14:19.803 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.0")) 2024-09-11T22:14:19.803 DEBUG:teuthology.orchestra.run.smithi092:> ' 2024-09-11T22:14:19.922 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.0.86294.asok 2024-09-11T22:14:19.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:14:19.932 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.86294.asok 2024-09-11T22:14:19.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:19.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.86294.asok status 2024-09-11T22:14:20.067 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:14:20.067 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2024-09-11T22:14:20.067 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-11T22:14:20.067 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:14:20.067 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "0", 2024-09-11T22:14:20.067 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2024-09-11T22:14:20.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "86294", 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/volumes/qa/sv_0/6d5615a6-07ce-4091-80b5-509f10920896" 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 15315, 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 15315 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-11T22:14:20.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 318816253 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 318816253 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:14:20.069 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.15315 192.168.0.1:0/318816253", 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/318816253", 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 5, 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 27, 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 94, 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2024-09-11T22:14:20.070 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:14:20.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:20.076 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2024-09-11T22:14:20.076 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:20.092 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2024-09-11T22:14:20.092 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:20.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:14:20.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:20.160 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T22:14:20.174 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:14:20.174 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T22:14:20.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 -- 172.21.15.92:0/3006644614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0100a60 msgr2=0x7ff3b0100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 --2- 172.21.15.92:0/3006644614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0100a60 0x7ff3b0100e60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009900 tx=0x7ff3ac02a410 comp rx=0 tx=0).stop 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 -- 172.21.15.92:0/3006644614 shutdown_connections 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 --2- 172.21.15.92:0/3006644614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3b0102610 0x7ff3b0108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 --2- 172.21.15.92:0/3006644614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3b0101c50 0x7ff3b01020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 --2- 172.21.15.92:0/3006644614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0100a60 0x7ff3b0100e60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 -- 172.21.15.92:0/3006644614 >> 172.21.15.92:0/3006644614 conn(0x7ff3b00fc1e0 msgr2=0x7ff3b00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 -- 172.21.15.92:0/3006644614 shutdown_connections 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.300+0000 7ff3b798d640 1 -- 172.21.15.92:0/3006644614 wait complete. 2024-09-11T22:14:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 Processor -- start 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 -- start start 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3b0100a60 0x7ff3b0196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3b0101c50 0x7ff3b0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0102610 0x7ff3b019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010c570 con 0x7ff3b0102610 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010c3f0 con 0x7ff3b0101c50 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b798d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010c6f0 con 0x7ff3b0100a60 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b5f2a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0102610 0x7ff3b019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b5f2a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0102610 0x7ff3b019e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47358/0 (socket says 172.21.15.92:47358) 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b5f2a640 1 -- 172.21.15.92:0/2620289646 learned_addr learned my addr 172.21.15.92:0/2620289646 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b4f28640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3b0101c50 0x7ff3b0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.301+0000 7ff3b5729640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3b0100a60 0x7ff3b0196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b5f2a640 1 -- 172.21.15.92:0/2620289646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3b0100a60 msgr2=0x7ff3b0196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b5f2a640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3b0100a60 0x7ff3b0196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b5f2a640 1 -- 172.21.15.92:0/2620289646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3b0101c50 msgr2=0x7ff3b0197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b5f2a640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3b0101c50 0x7ff3b0197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b5f2a640 1 -- 172.21.15.92:0/2620289646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b019e850 con 0x7ff3b0102610 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b5729640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3b0100a60 0x7ff3b0196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b5f2a640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0102610 0x7ff3b019e1b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800eaa0 tx=0x7ff3a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3a67fc640 1 -- 172.21.15.92:0/2620289646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800cdb0 con 0x7ff3b0102610 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3a67fc640 1 -- 172.21.15.92:0/2620289646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a8005020 con 0x7ff3b0102610 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b019eb40 con 0x7ff3b0102610 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3a67fc640 1 -- 172.21.15.92:0/2620289646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80116d0 con 0x7ff3b0102610 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.302+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b019f090 con 0x7ff3b0102610 2024-09-11T22:14:20.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.303+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b0107ef0 con 0x7ff3b0102610 2024-09-11T22:14:20.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.304+0000 7ff3a67fc640 1 -- 172.21.15.92:0/2620289646 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff3a80051d0 con 0x7ff3b0102610 2024-09-11T22:14:20.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.305+0000 7ff3a67fc640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff39c0a2cb0 0x7ff39c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:20.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.305+0000 7ff3b5729640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff39c0a2cb0 0x7ff39c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:20.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.305+0000 7ff3a67fc640 1 -- 172.21.15.92:0/2620289646 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff3a80cf600 con 0x7ff3b0102610 2024-09-11T22:14:20.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.305+0000 7ff3b5729640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff39c0a2cb0 0x7ff39c0a5170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac006010 tx=0x7ff3ac032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.309+0000 7ff3a67fc640 1 -- 172.21.15.92:0/2620289646 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3a8016030 con 0x7ff3b0102610 2024-09-11T22:14:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.511+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff3b0062df0 con 0x7ff3b0102610 2024-09-11T22:14:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.785+0000 7ff3a67fc640 1 -- 172.21.15.92:0/2620289646 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7ff3a809c1d0 con 0x7ff3b0102610 2024-09-11T22:14:20.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff39c0a2cb0 msgr2=0x7ff39c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff39c0a2cb0 0x7ff39c0a5170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac006010 tx=0x7ff3ac032d20 comp rx=0 tx=0).stop 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0102610 msgr2=0x7ff3b019e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0102610 0x7ff3b019e1b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800eaa0 tx=0x7ff3a800ef60 comp rx=0 tx=0).stop 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 shutdown_connections 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7ff39c0a2cb0 0x7ff39c0a5170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0102610 0x7ff3b019e1b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3b0101c50 0x7ff3b0197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 --2- 172.21.15.92:0/2620289646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3b0100a60 0x7ff3b0196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 >> 172.21.15.92:0/2620289646 conn(0x7ff3b00fc1e0 msgr2=0x7ff3b01050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 shutdown_connections 2024-09-11T22:14:20.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:14:20.788+0000 7ff3b798d640 1 -- 172.21.15.92:0/2620289646 wait complete. 2024-09-11T22:14:20.802 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T22:14:20.820 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T22:14:20.842 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T22:14:20.843 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T22:14:20.843 INFO:tasks.workunit:timeout=3h 2024-09-11T22:14:20.843 INFO:tasks.workunit:cleanup=True 2024-09-11T22:14:20.844 DEBUG:teuthology.orchestra.run.smithi092:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 22:12:43.497093864 +0000 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 22:12:43.497093864 +0000 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 22:14:20.156814068 +0000 2024-09-11T22:14:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-11T22:14:20.861 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T22:14:20.862 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T22:14:20.934 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T22:14:20.991 INFO:tasks.workunit.client.0.smithi092.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T22:14:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:20 smithi110 ceph-mon[34830]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-11T22:14:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:20 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2620289646' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:14:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:20 smithi092 ceph-mon[31173]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-11T22:14:21.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:20 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2620289646' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:14:21.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:20 smithi153 ceph-mon[35045]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-11T22:14:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:20 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2620289646' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:14:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:21 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2620289646' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T22:14:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:21 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:14:22.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:21 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2620289646' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T22:14:22.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:21 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:14:22.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:21 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2620289646' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T22:14:22.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:21 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:14:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:22 smithi110 ceph-mon[34830]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:22 smithi092 ceph-mon[31173]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:22 smithi153 ceph-mon[35045]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:25.070 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:24 smithi092 ceph-mon[31173]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:24 smithi110 ceph-mon[34830]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:25.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:24 smithi153 ceph-mon[35045]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:27.091 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:26 smithi153 ceph-mon[35045]: pgmap v359: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.0 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:26 smithi110 ceph-mon[34830]: pgmap v359: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.0 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:27.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:26 smithi092 ceph-mon[31173]: pgmap v359: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.0 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:28 smithi110 ceph-mon[34830]: pgmap v360: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:29.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:28 smithi092 ceph-mon[31173]: pgmap v360: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:29.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:28 smithi153 ceph-mon[35045]: pgmap v360: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:30 smithi110 ceph-mon[34830]: pgmap v361: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:30 smithi092 ceph-mon[31173]: pgmap v361: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:30 smithi153 ceph-mon[35045]: pgmap v361: 129 pgs: 129 active+clean; 655 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:32 smithi110 ceph-mon[34830]: pgmap v362: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:33.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:32 smithi092 ceph-mon[31173]: pgmap v362: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:33.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:32 smithi153 ceph-mon[35045]: pgmap v362: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T22:14:35.126 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:34 smithi092 ceph-mon[31173]: pgmap v363: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-11T22:14:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:34 smithi110 ceph-mon[34830]: pgmap v363: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-11T22:14:35.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:34 smithi153 ceph-mon[35045]: pgmap v363: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-11T22:14:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:36 smithi110 ceph-mon[34830]: pgmap v364: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T22:14:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:36 smithi092 ceph-mon[31173]: pgmap v364: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T22:14:37.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:36 smithi153 ceph-mon[35045]: pgmap v364: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T22:14:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:38 smithi110 ceph-mon[34830]: pgmap v365: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T22:14:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:38 smithi092 ceph-mon[31173]: pgmap v365: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T22:14:39.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:38 smithi153 ceph-mon[35045]: pgmap v365: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T22:14:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:40 smithi110 ceph-mon[34830]: pgmap v366: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T22:14:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:40 smithi092 ceph-mon[31173]: pgmap v366: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T22:14:41.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:40 smithi153 ceph-mon[35045]: pgmap v366: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T22:14:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:42 smithi110 ceph-mon[34830]: pgmap v367: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T22:14:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:42 smithi092 ceph-mon[31173]: pgmap v367: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T22:14:43.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:42 smithi153 ceph-mon[35045]: pgmap v367: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T22:14:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:44 smithi110 ceph-mon[34830]: pgmap v368: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:44 smithi092 ceph-mon[31173]: pgmap v368: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:45.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:44 smithi153 ceph-mon[35045]: pgmap v368: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:46 smithi110 ceph-mon[34830]: pgmap v369: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:46 smithi092 ceph-mon[31173]: pgmap v369: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:46 smithi153 ceph-mon[35045]: pgmap v369: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:48 smithi110 ceph-mon[34830]: pgmap v370: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:48 smithi092 ceph-mon[31173]: pgmap v370: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:49.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:48 smithi153 ceph-mon[35045]: pgmap v370: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:50 smithi110 ceph-mon[34830]: pgmap v371: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:50 smithi092 ceph-mon[31173]: pgmap v371: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:50 smithi153 ceph-mon[35045]: pgmap v371: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:14:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:52 smithi110 ceph-mon[34830]: pgmap v372: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:52 smithi092 ceph-mon[31173]: pgmap v372: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:53.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:52 smithi153 ceph-mon[35045]: pgmap v372: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:14:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:54 smithi110 ceph-mon[34830]: pgmap v373: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:54 smithi092 ceph-mon[31173]: pgmap v373: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:55.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:54 smithi153 ceph-mon[35045]: pgmap v373: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:14:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:56 smithi110 ceph-mon[34830]: pgmap v374: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:56 smithi092 ceph-mon[31173]: pgmap v374: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:57.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:56 smithi153 ceph-mon[35045]: pgmap v374: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:14:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:14:58 smithi110 ceph-mon[34830]: pgmap v375: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:59.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:14:58 smithi092 ceph-mon[31173]: pgmap v375: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:14:59.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:14:58 smithi153 ceph-mon[35045]: pgmap v375: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:15:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:00 smithi110 ceph-mon[34830]: pgmap v376: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:15:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:00 smithi092 ceph-mon[31173]: pgmap v376: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:15:01.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:00 smithi153 ceph-mon[35045]: pgmap v376: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:15:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:02 smithi110 ceph-mon[34830]: pgmap v377: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:15:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:02 smithi092 ceph-mon[31173]: pgmap v377: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:15:03.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:02 smithi153 ceph-mon[35045]: pgmap v377: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:15:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:04 smithi110 ceph-mon[34830]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:15:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:05.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:04 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:15:05.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:04 smithi092 ceph-mon[31173]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:15:05.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:05.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:05.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:05.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:04 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:15:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:04 smithi153 ceph-mon[35045]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:15:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:04 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:15:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:06 smithi110 ceph-mon[34830]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:15:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:06 smithi092 ceph-mon[31173]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:15:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:06 smithi153 ceph-mon[35045]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:15:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:08 smithi110 ceph-mon[34830]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:15:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:08 smithi092 ceph-mon[31173]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:15:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:08 smithi153 ceph-mon[35045]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:15:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:10 smithi110 ceph-mon[34830]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:15:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:10 smithi092 ceph-mon[31173]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:15:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:10 smithi153 ceph-mon[35045]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-11T22:15:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:12 smithi110 ceph-mon[34830]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:15:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:12 smithi092 ceph-mon[31173]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:15:13.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:12 smithi153 ceph-mon[35045]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T22:15:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:14 smithi110 ceph-mon[34830]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:15:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:14 smithi092 ceph-mon[31173]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:15:15.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:14 smithi153 ceph-mon[35045]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:15:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:16 smithi110 ceph-mon[34830]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:15:17.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:16 smithi092 ceph-mon[31173]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:15:17.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:16 smithi153 ceph-mon[35045]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-11T22:15:17.390 INFO:tasks.workunit.client.0.smithi092.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-11T22:15:17.390 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:17.390 INFO:tasks.workunit.client.0.smithi092.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T22:15:17.390 INFO:tasks.workunit.client.0.smithi092.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T22:15:17.390 INFO:tasks.workunit.client.0.smithi092.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T22:15:17.390 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:17.390 INFO:tasks.workunit.client.0.smithi092.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr: git switch -c 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr:Or undo this operation with: 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr: git switch - 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:17.391 INFO:tasks.workunit.client.0.smithi092.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-11T22:15:17.396 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T22:15:17.458 INFO:tasks.workunit.client.0.smithi092.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T22:15:17.460 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T22:15:17.460 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T22:15:18.168 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T22:15:18.236 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T22:15:18.277 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T22:15:18.278 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T22:15:18.278 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T22:15:18.317 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T22:15:18.321 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:15:18.321 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T22:15:18.377 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-11T22:15:18.379 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-11T22:15:18.379 DEBUG:teuthology.orchestra.run.smithi092:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-11T22:15:18.442 INFO:tasks.workunit.client.0.smithi092.stdout:getting blogbench 2024-09-11T22:15:18.442 INFO:tasks.workunit.client.0.smithi092.stderr:+ echo 'getting blogbench' 2024-09-11T22:15:18.442 INFO:tasks.workunit.client.0.smithi092.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-11T22:15:18.539 INFO:tasks.workunit.client.0.smithi092.stderr:--2024-09-11 22:15:18-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-11T22:15:18.540 INFO:tasks.workunit.client.0.smithi092.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-11T22:15:18.561 INFO:tasks.workunit.client.0.smithi092.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-11T22:15:18.582 INFO:tasks.workunit.client.0.smithi092.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-11T22:15:18.583 INFO:tasks.workunit.client.0.smithi092.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-11T22:15:18.584 INFO:tasks.workunit.client.0.smithi092.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-11T22:15:18.588 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:18.624 INFO:tasks.workunit.client.0.smithi092.stderr: 0K .......... .......... .......... .......... .......... 49% 1.25M 0s 2024-09-11T22:15:18.644 INFO:tasks.workunit.client.0.smithi092.stderr: 50K .......... .......... .......... .......... .......... 99% 2.41M 0s 2024-09-11T22:15:18.644 INFO:tasks.workunit.client.0.smithi092.stderr: 100K 100% 1.74T=0.06s 2024-09-11T22:15:18.644 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:18.645 INFO:tasks.workunit.client.0.smithi092.stderr:2024-09-11 22:15:18 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-11T22:15:18.645 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-11T22:15:18.645 INFO:tasks.workunit.client.0.smithi092.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-11T22:15:18.689 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/ 2024-09-11T22:15:18.690 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/m4/ 2024-09-11T22:15:18.692 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-11T22:15:18.694 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/man/ 2024-09-11T22:15:18.696 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/man/blogbench.8 2024-09-11T22:15:18.698 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/man/Makefile.am 2024-09-11T22:15:18.700 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/man/Makefile.in 2024-09-11T22:15:18.702 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/ 2024-09-11T22:15:18.704 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-11T22:15:18.706 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-11T22:15:18.708 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/blogbench.c 2024-09-11T22:15:18.710 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/blogbench.h 2024-09-11T22:15:18.712 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-11T22:15:18.715 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-11T22:15:18.717 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-11T22:15:18.719 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-11T22:15:18.722 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-11T22:15:18.724 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/Makefile.am 2024-09-11T22:15:18.726 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/Makefile.in 2024-09-11T22:15:18.728 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/process.c 2024-09-11T22:15:18.731 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/globals.h 2024-09-11T22:15:18.732 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/commenter.c 2024-09-11T22:15:18.734 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/rewriter.c 2024-09-11T22:15:18.736 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/reader.c 2024-09-11T22:15:18.739 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/writer.c 2024-09-11T22:15:18.741 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/src/helpers.c 2024-09-11T22:15:18.742 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/NEWS 2024-09-11T22:15:18.744 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/compile 2024-09-11T22:15:18.746 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/depcomp 2024-09-11T22:15:18.749 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/aclocal.m4 2024-09-11T22:15:18.751 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/README 2024-09-11T22:15:18.753 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/configure 2024-09-11T22:15:18.761 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/configure.ac 2024-09-11T22:15:18.763 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/config.guess 2024-09-11T22:15:18.766 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/install-sh 2024-09-11T22:15:18.768 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/config.sub 2024-09-11T22:15:18.771 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/missing 2024-09-11T22:15:18.774 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/mkinstalldirs 2024-09-11T22:15:18.776 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/Makefile.am 2024-09-11T22:15:18.778 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/Makefile.in 2024-09-11T22:15:18.780 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/config.h.in 2024-09-11T22:15:18.782 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/AUTHORS 2024-09-11T22:15:18.785 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/INSTALL 2024-09-11T22:15:18.787 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/ChangeLog 2024-09-11T22:15:18.789 INFO:tasks.workunit.client.0.smithi092.stdout:blogbench-1.0/COPYING 2024-09-11T22:15:18.791 INFO:tasks.workunit.client.0.smithi092.stderr:+ cd blogbench-1.0/ 2024-09-11T22:15:18.792 INFO:tasks.workunit.client.0.smithi092.stdout:making blogbench 2024-09-11T22:15:18.793 INFO:tasks.workunit.client.0.smithi092.stderr:+ echo 'making blogbench' 2024-09-11T22:15:18.793 INFO:tasks.workunit.client.0.smithi092.stderr:+ ./configure 2024-09-11T22:15:18.901 INFO:tasks.workunit.client.0.smithi092.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-11T22:15:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:18 smithi110 ceph-mon[34830]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:15:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:18 smithi092 ceph-mon[31173]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:15:19.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:18 smithi153 ceph-mon[35045]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:15:19.912 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether build environment is sane... yes 2024-09-11T22:15:19.928 INFO:tasks.workunit.client.0.smithi092.stdout:checking for gawk... gawk 2024-09-11T22:15:19.950 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether make sets $(MAKE)... yes 2024-09-11T22:15:19.966 INFO:tasks.workunit.client.0.smithi092.stdout:checking for gcc... gcc 2024-09-11T22:15:20.039 INFO:tasks.workunit.client.0.smithi092.stdout:checking for C compiler default output file name... a.out 2024-09-11T22:15:20.041 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether the C compiler works... yes 2024-09-11T22:15:20.045 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether we are cross compiling... no 2024-09-11T22:15:20.092 INFO:tasks.workunit.client.0.smithi092.stdout:checking for suffix of executables... 2024-09-11T22:15:20.135 INFO:tasks.workunit.client.0.smithi092.stdout:checking for suffix of object files... o 2024-09-11T22:15:20.176 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether we are using the GNU C compiler... yes 2024-09-11T22:15:20.220 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether gcc accepts -g... yes 2024-09-11T22:15:20.294 INFO:tasks.workunit.client.0.smithi092.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-11T22:15:20.355 INFO:tasks.workunit.client.0.smithi092.stdout:checking for style of include used by make... GNU 2024-09-11T22:15:20.491 INFO:tasks.workunit.client.0.smithi092.stdout:checking dependency style of gcc... gcc3 2024-09-11T22:15:20.492 INFO:tasks.workunit.client.0.smithi092.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-11T22:15:20.571 INFO:tasks.workunit.client.0.smithi092.stdout:checking how to run the C preprocessor... gcc -E 2024-09-11T22:15:20.629 INFO:tasks.workunit.client.0.smithi092.stdout:checking for egrep... grep -E 2024-09-11T22:15:20.642 INFO:tasks.workunit.client.0.smithi092.stdout:checking for AIX... no 2024-09-11T22:15:20.731 INFO:tasks.workunit.client.0.smithi092.stdout:checking for library containing strerror... none required 2024-09-11T22:15:20.922 INFO:tasks.workunit.client.0.smithi092.stdout:checking for ANSI C header files... yes 2024-09-11T22:15:20.971 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-11T22:15:21.032 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sys/types.h... yes 2024-09-11T22:15:21.097 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sys/stat.h... yes 2024-09-11T22:15:21.160 INFO:tasks.workunit.client.0.smithi092.stdout:checking for stdlib.h... yes 2024-09-11T22:15:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:20 smithi110 ceph-mon[34830]: pgmap v386: 129 pgs: 129 active+clean; 715 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-11T22:15:21.223 INFO:tasks.workunit.client.0.smithi092.stdout:checking for string.h... yes 2024-09-11T22:15:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:20 smithi092 ceph-mon[31173]: pgmap v386: 129 pgs: 129 active+clean; 715 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-11T22:15:21.286 INFO:tasks.workunit.client.0.smithi092.stdout:checking for memory.h... yes 2024-09-11T22:15:21.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:20 smithi153 ceph-mon[35045]: pgmap v386: 129 pgs: 129 active+clean; 715 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-11T22:15:21.348 INFO:tasks.workunit.client.0.smithi092.stdout:checking for strings.h... yes 2024-09-11T22:15:21.411 INFO:tasks.workunit.client.0.smithi092.stdout:checking for inttypes.h... yes 2024-09-11T22:15:21.474 INFO:tasks.workunit.client.0.smithi092.stdout:checking for stdint.h... yes 2024-09-11T22:15:21.543 INFO:tasks.workunit.client.0.smithi092.stdout:checking for unistd.h... yes 2024-09-11T22:15:21.560 INFO:tasks.workunit.client.0.smithi092.stdout:checking for memory.h... (cached) yes 2024-09-11T22:15:21.571 INFO:tasks.workunit.client.0.smithi092.stdout:checking for strings.h... (cached) yes 2024-09-11T22:15:21.583 INFO:tasks.workunit.client.0.smithi092.stdout:checking for unistd.h... (cached) yes 2024-09-11T22:15:21.658 INFO:tasks.workunit.client.0.smithi092.stdout:checking locale.h usability... yes 2024-09-11T22:15:21.683 INFO:tasks.workunit.client.0.smithi092.stdout:checking locale.h presence... yes 2024-09-11T22:15:21.684 INFO:tasks.workunit.client.0.smithi092.stdout:checking for locale.h... yes 2024-09-11T22:15:21.704 INFO:tasks.workunit.client.0.smithi092.stdout:checking for stdlib.h... (cached) yes 2024-09-11T22:15:21.715 INFO:tasks.workunit.client.0.smithi092.stdout:checking for string.h... (cached) yes 2024-09-11T22:15:21.787 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/socket.h usability... yes 2024-09-11T22:15:21.816 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/socket.h presence... yes 2024-09-11T22:15:21.817 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sys/socket.h... yes 2024-09-11T22:15:21.898 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/time.h usability... yes 2024-09-11T22:15:21.926 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/time.h presence... yes 2024-09-11T22:15:21.927 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sys/time.h... yes 2024-09-11T22:15:21.999 INFO:tasks.workunit.client.0.smithi092.stdout:checking fcntl.h usability... yes 2024-09-11T22:15:22.030 INFO:tasks.workunit.client.0.smithi092.stdout:checking fcntl.h presence... yes 2024-09-11T22:15:22.031 INFO:tasks.workunit.client.0.smithi092.stdout:checking for fcntl.h... yes 2024-09-11T22:15:22.100 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/fcntl.h usability... yes 2024-09-11T22:15:22.127 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/fcntl.h presence... yes 2024-09-11T22:15:22.128 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sys/fcntl.h... yes 2024-09-11T22:15:22.234 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/param.h usability... yes 2024-09-11T22:15:22.262 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/param.h presence... yes 2024-09-11T22:15:22.263 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sys/param.h... yes 2024-09-11T22:15:22.323 INFO:tasks.workunit.client.0.smithi092.stdout:checking ioctl.h usability... no 2024-09-11T22:15:22.352 INFO:tasks.workunit.client.0.smithi092.stdout:checking ioctl.h presence... no 2024-09-11T22:15:22.353 INFO:tasks.workunit.client.0.smithi092.stdout:checking for ioctl.h... no 2024-09-11T22:15:22.416 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/ioctl.h usability... yes 2024-09-11T22:15:22.444 INFO:tasks.workunit.client.0.smithi092.stdout:checking sys/ioctl.h presence... yes 2024-09-11T22:15:22.444 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sys/ioctl.h... yes 2024-09-11T22:15:22.506 INFO:tasks.workunit.client.0.smithi092.stdout:checking return type of signal handlers... void 2024-09-11T22:15:22.567 INFO:tasks.workunit.client.0.smithi092.stdout:checking for size_t... yes 2024-09-11T22:15:22.616 INFO:tasks.workunit.client.0.smithi092.stdout:checking for dev_t... yes 2024-09-11T22:15:22.669 INFO:tasks.workunit.client.0.smithi092.stdout:checking for short... yes 2024-09-11T22:15:22.764 INFO:tasks.workunit.client.0.smithi092.stdout:checking size of short... 2 2024-09-11T22:15:22.821 INFO:tasks.workunit.client.0.smithi092.stdout:checking for int... yes 2024-09-11T22:15:22.930 INFO:tasks.workunit.client.0.smithi092.stdout:checking size of int... 4 2024-09-11T22:15:22.988 INFO:tasks.workunit.client.0.smithi092.stdout:checking for long... yes 2024-09-11T22:15:23.074 INFO:tasks.workunit.client.0.smithi092.stdout:checking size of long... 8 2024-09-11T22:15:23.129 INFO:tasks.workunit.client.0.smithi092.stdout:checking for long long... yes 2024-09-11T22:15:23.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:22 smithi110 ceph-mon[34830]: pgmap v387: 129 pgs: 129 active+clean; 859 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-11T22:15:23.217 INFO:tasks.workunit.client.0.smithi092.stdout:checking size of long long... 8 2024-09-11T22:15:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:22 smithi092 ceph-mon[31173]: pgmap v387: 129 pgs: 129 active+clean; 859 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-11T22:15:23.275 INFO:tasks.workunit.client.0.smithi092.stdout:checking for ssize_t... yes 2024-09-11T22:15:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:22 smithi153 ceph-mon[35045]: pgmap v387: 129 pgs: 129 active+clean; 859 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-11T22:15:23.323 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sig_atomic_t... yes 2024-09-11T22:15:23.380 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether gcc needs -traditional... no 2024-09-11T22:15:23.427 INFO:tasks.workunit.client.0.smithi092.stdout:checking for an ANSI C-conforming const... yes 2024-09-11T22:15:23.513 INFO:tasks.workunit.client.0.smithi092.stdout:checking for inline... inline 2024-09-11T22:15:23.596 INFO:tasks.workunit.client.0.smithi092.stdout:checking for working alloca.h... yes 2024-09-11T22:15:23.662 INFO:tasks.workunit.client.0.smithi092.stdout:checking for alloca... yes 2024-09-11T22:15:23.745 INFO:tasks.workunit.client.0.smithi092.stdout:checking for working memcmp... yes 2024-09-11T22:15:23.823 INFO:tasks.workunit.client.0.smithi092.stdout:checking for strftime... yes 2024-09-11T22:15:23.937 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-11T22:15:24.018 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether stat accepts an empty string... no 2024-09-11T22:15:24.097 INFO:tasks.workunit.client.0.smithi092.stdout:checking for vprintf... yes 2024-09-11T22:15:24.175 INFO:tasks.workunit.client.0.smithi092.stdout:checking for _doprnt... no 2024-09-11T22:15:24.203 INFO:tasks.workunit.client.0.smithi092.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-11T22:15:24.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:23 smithi110 ceph-mon[34830]: pgmap v388: 129 pgs: 129 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-09-11T22:15:24.217 INFO:tasks.workunit.client.0.smithi092.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-11T22:15:24.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:23 smithi092 ceph-mon[31173]: pgmap v388: 129 pgs: 129 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-09-11T22:15:24.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:23 smithi153 ceph-mon[35045]: pgmap v388: 129 pgs: 129 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-09-11T22:15:24.293 INFO:tasks.workunit.client.0.smithi092.stdout:checking for the pthreads library -lpthreads... no 2024-09-11T22:15:24.363 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether pthreads work without any flags... yes 2024-09-11T22:15:24.426 INFO:tasks.workunit.client.0.smithi092.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-11T22:15:24.426 INFO:tasks.workunit.client.0.smithi092.stdout:checking if more special flags are required for pthreads... no 2024-09-11T22:15:24.427 INFO:tasks.workunit.client.0.smithi092.stdout:checking for cc_r... gcc 2024-09-11T22:15:24.509 INFO:tasks.workunit.client.0.smithi092.stdout:checking for setrlimit... yes 2024-09-11T22:15:24.604 INFO:tasks.workunit.client.0.smithi092.stdout:checking for setproctitle... no 2024-09-11T22:15:24.699 INFO:tasks.workunit.client.0.smithi092.stdout:checking for getopt_long... yes 2024-09-11T22:15:24.782 INFO:tasks.workunit.client.0.smithi092.stdout:checking for strtoull... yes 2024-09-11T22:15:24.865 INFO:tasks.workunit.client.0.smithi092.stdout:checking for strtoq... yes 2024-09-11T22:15:24.965 INFO:tasks.workunit.client.0.smithi092.stdout:checking for snprintf... yes 2024-09-11T22:15:25.051 INFO:tasks.workunit.client.0.smithi092.stdout:checking for vsnprintf... yes 2024-09-11T22:15:25.135 INFO:tasks.workunit.client.0.smithi092.stdout:checking for vfprintf... yes 2024-09-11T22:15:25.219 INFO:tasks.workunit.client.0.smithi092.stdout:checking for gethostname... yes 2024-09-11T22:15:25.298 INFO:tasks.workunit.client.0.smithi092.stdout:checking for setlocale... yes 2024-09-11T22:15:25.379 INFO:tasks.workunit.client.0.smithi092.stdout:checking for sync... yes 2024-09-11T22:15:25.462 INFO:tasks.workunit.client.0.smithi092.stdout:checking whether snprintf is C99 conformant... done 2024-09-11T22:15:25.479 INFO:tasks.workunit.client.0.smithi092.stdout:configure: creating ./config.status 2024-09-11T22:15:25.701 INFO:tasks.workunit.client.0.smithi092.stdout:config.status: creating Makefile 2024-09-11T22:15:25.723 INFO:tasks.workunit.client.0.smithi092.stdout:config.status: creating src/Makefile 2024-09-11T22:15:25.748 INFO:tasks.workunit.client.0.smithi092.stdout:config.status: creating man/Makefile 2024-09-11T22:15:25.764 INFO:tasks.workunit.client.0.smithi092.stdout:config.status: creating config.h 2024-09-11T22:15:25.832 INFO:tasks.workunit.client.0.smithi092.stdout:config.status: executing depfiles commands 2024-09-11T22:15:26.016 INFO:tasks.workunit.client.0.smithi092.stderr:+ make 2024-09-11T22:15:26.029 INFO:tasks.workunit.client.0.smithi092.stdout:make all-recursive 2024-09-11T22:15:26.036 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T22:15:26.039 INFO:tasks.workunit.client.0.smithi092.stdout:Making all in src 2024-09-11T22:15:26.061 INFO:tasks.workunit.client.0.smithi092.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-11T22:15:26.061 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-11T22:15:26.062 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-11T22:15:26.169 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-11T22:15:26.169 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-11T22:15:26.318 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-11T22:15:26.319 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-11T22:15:26.448 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-11T22:15:26.449 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-11T22:15:26.581 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-11T22:15:26.581 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-11T22:15:26.719 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-11T22:15:26.719 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-11T22:15:26.850 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-11T22:15:26.850 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-11T22:15:27.021 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-11T22:15:27.021 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-11T22:15:27.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:26 smithi153 ceph-mon[35045]: pgmap v389: 129 pgs: 129 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-09-11T22:15:27.138 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-11T22:15:27.139 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-11T22:15:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:26 smithi110 ceph-mon[34830]: pgmap v389: 129 pgs: 129 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-09-11T22:15:27.247 INFO:tasks.workunit.client.0.smithi092.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-11T22:15:27.247 INFO:tasks.workunit.client.0.smithi092.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-11T22:15:27.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:26 smithi092 ceph-mon[31173]: pgmap v389: 129 pgs: 129 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-09-11T22:15:27.331 INFO:tasks.workunit.client.0.smithi092.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-11T22:15:27.390 INFO:tasks.workunit.client.0.smithi092.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-11T22:15:27.394 INFO:tasks.workunit.client.0.smithi092.stdout:Making all in man 2024-09-11T22:15:27.403 INFO:tasks.workunit.client.0.smithi092.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-11T22:15:27.404 INFO:tasks.workunit.client.0.smithi092.stdout:make[2]: Nothing to be done for 'all'. 2024-09-11T22:15:27.404 INFO:tasks.workunit.client.0.smithi092.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-11T22:15:27.411 INFO:tasks.workunit.client.0.smithi092.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T22:15:27.413 INFO:tasks.workunit.client.0.smithi092.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T22:15:27.414 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-11T22:15:27.414 INFO:tasks.workunit.client.0.smithi092.stderr:+ cd src 2024-09-11T22:15:27.416 INFO:tasks.workunit.client.0.smithi092.stderr:+ mkdir blogtest_in 2024-09-11T22:15:27.418 INFO:tasks.workunit.client.0.smithi092.stdout:running blogbench 2024-09-11T22:15:27.419 INFO:tasks.workunit.client.0.smithi092.stderr:+ echo 'running blogbench' 2024-09-11T22:15:27.419 INFO:tasks.workunit.client.0.smithi092.stderr:+ ./blogbench -d blogtest_in 2024-09-11T22:15:27.419 INFO:tasks.workunit.client.0.smithi092.stdout: 2024-09-11T22:15:27.419 INFO:tasks.workunit.client.0.smithi092.stdout:Frequency = 10 secs 2024-09-11T22:15:27.419 INFO:tasks.workunit.client.0.smithi092.stdout:Scratch dir = [blogtest_in] 2024-09-11T22:15:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:28 smithi110 ceph-mon[34830]: pgmap v390: 129 pgs: 129 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 356 KiB/s wr, 64 op/s 2024-09-11T22:15:29.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:28 smithi092 ceph-mon[31173]: pgmap v390: 129 pgs: 129 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 356 KiB/s wr, 64 op/s 2024-09-11T22:15:29.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:28 smithi153 ceph-mon[35045]: pgmap v390: 129 pgs: 129 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 356 KiB/s wr, 64 op/s 2024-09-11T22:15:30.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:29 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:30.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:29 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:30.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:29 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:30.609 INFO:tasks.workunit.client.0.smithi092.stdout:Spawning 3 writers... 2024-09-11T22:15:30.610 INFO:tasks.workunit.client.0.smithi092.stdout:Spawning 1 rewriters... 2024-09-11T22:15:30.610 INFO:tasks.workunit.client.0.smithi092.stdout:Spawning 5 commenters... 2024-09-11T22:15:30.610 INFO:tasks.workunit.client.0.smithi092.stdout:Spawning 100 readers... 2024-09-11T22:15:30.612 INFO:tasks.workunit.client.0.smithi092.stdout:Benchmarking for 30 iterations. 2024-09-11T22:15:30.612 INFO:tasks.workunit.client.0.smithi092.stdout:The test will run during 5 minutes. 2024-09-11T22:15:30.612 INFO:tasks.workunit.client.0.smithi092.stdout: 2024-09-11T22:15:30.612 INFO:tasks.workunit.client.0.smithi092.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-11T22:15:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:30 smithi110 ceph-mon[34830]: pgmap v391: 129 pgs: 129 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 442 KiB/s wr, 126 op/s 2024-09-11T22:15:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:30 smithi092 ceph-mon[31173]: pgmap v391: 129 pgs: 129 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 442 KiB/s wr, 126 op/s 2024-09-11T22:15:31.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:30 smithi153 ceph-mon[35045]: pgmap v391: 129 pgs: 129 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 442 KiB/s wr, 126 op/s 2024-09-11T22:15:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:32 smithi110 ceph-mon[34830]: pgmap v392: 129 pgs: 129 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 536 KiB/s wr, 135 op/s 2024-09-11T22:15:33.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:32 smithi092 ceph-mon[31173]: pgmap v392: 129 pgs: 129 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 536 KiB/s wr, 135 op/s 2024-09-11T22:15:33.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:32 smithi153 ceph-mon[35045]: pgmap v392: 129 pgs: 129 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 536 KiB/s wr, 135 op/s 2024-09-11T22:15:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:34 smithi110 ceph-mon[34830]: pgmap v393: 129 pgs: 129 active+clean; 14 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-09-11T22:15:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:34 smithi092 ceph-mon[31173]: pgmap v393: 129 pgs: 129 active+clean; 14 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-09-11T22:15:35.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:34 smithi153 ceph-mon[35045]: pgmap v393: 129 pgs: 129 active+clean; 14 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-09-11T22:15:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:36 smithi110 ceph-mon[34830]: pgmap v394: 129 pgs: 129 active+clean; 17 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:15:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:36 smithi092 ceph-mon[31173]: pgmap v394: 129 pgs: 129 active+clean; 17 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:15:37.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:36 smithi153 ceph-mon[35045]: pgmap v394: 129 pgs: 129 active+clean; 17 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:15:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:38 smithi110 ceph-mon[34830]: pgmap v395: 129 pgs: 129 active+clean; 17 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-09-11T22:15:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:38 smithi092 ceph-mon[31173]: pgmap v395: 129 pgs: 129 active+clean; 17 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-09-11T22:15:39.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:38 smithi153 ceph-mon[35045]: pgmap v395: 129 pgs: 129 active+clean; 17 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-09-11T22:15:40.612 INFO:tasks.workunit.client.0.smithi092.stdout: 6 13471 463 6376 259 4367 519 2024-09-11T22:15:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:40 smithi110 ceph-mon[34830]: pgmap v396: 129 pgs: 129 active+clean; 27 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-11T22:15:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:40 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:40 smithi092 ceph-mon[31173]: pgmap v396: 129 pgs: 129 active+clean; 27 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-11T22:15:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:40 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:41.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:40 smithi153 ceph-mon[35045]: pgmap v396: 129 pgs: 129 active+clean; 27 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-11T22:15:41.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:40 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:42 smithi110 ceph-mon[34830]: pgmap v397: 129 pgs: 129 active+clean; 28 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-09-11T22:15:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:42 smithi092 ceph-mon[31173]: pgmap v397: 129 pgs: 129 active+clean; 28 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-09-11T22:15:43.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:42 smithi153 ceph-mon[35045]: pgmap v397: 129 pgs: 129 active+clean; 28 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-09-11T22:15:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:44 smithi110 ceph-mon[34830]: pgmap v398: 129 pgs: 129 active+clean; 45 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-09-11T22:15:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:44 smithi092 ceph-mon[31173]: pgmap v398: 129 pgs: 129 active+clean; 45 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-09-11T22:15:45.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:44 smithi153 ceph-mon[35045]: pgmap v398: 129 pgs: 129 active+clean; 45 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-09-11T22:15:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:45 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:46.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:45 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:46.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:45 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:46 smithi110 ceph-mon[34830]: pgmap v399: 129 pgs: 129 active+clean; 52 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-11T22:15:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:46 smithi092 ceph-mon[31173]: pgmap v399: 129 pgs: 129 active+clean; 52 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-11T22:15:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:46 smithi153 ceph-mon[35045]: pgmap v399: 129 pgs: 129 active+clean; 52 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-11T22:15:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:48 smithi110 ceph-mon[34830]: pgmap v400: 129 pgs: 129 active+clean; 56 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 3.5 MiB/s wr, 197 op/s 2024-09-11T22:15:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:48 smithi092 ceph-mon[31173]: pgmap v400: 129 pgs: 129 active+clean; 56 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 3.5 MiB/s wr, 197 op/s 2024-09-11T22:15:49.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:48 smithi153 ceph-mon[35045]: pgmap v400: 129 pgs: 129 active+clean; 56 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 3.5 MiB/s wr, 197 op/s 2024-09-11T22:15:50.612 INFO:tasks.workunit.client.0.smithi092.stdout:write(): Bad file descriptor 23 10120 853 5014 787 1955 70 2024-09-11T22:15:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:50 smithi110 ceph-mon[34830]: pgmap v401: 129 pgs: 129 active+clean; 75 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-11T22:15:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:50 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:50 smithi092 ceph-mon[31173]: pgmap v401: 129 pgs: 129 active+clean; 75 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-11T22:15:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:50 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:51.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:50 smithi153 ceph-mon[35045]: pgmap v401: 129 pgs: 129 active+clean; 75 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-11T22:15:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:50 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:15:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:52 smithi110 ceph-mon[34830]: pgmap v402: 129 pgs: 129 active+clean; 77 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 4.4 MiB/s wr, 216 op/s 2024-09-11T22:15:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:52 smithi092 ceph-mon[31173]: pgmap v402: 129 pgs: 129 active+clean; 77 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 4.4 MiB/s wr, 216 op/s 2024-09-11T22:15:53.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:52 smithi153 ceph-mon[35045]: pgmap v402: 129 pgs: 129 active+clean; 77 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 4.4 MiB/s wr, 216 op/s 2024-09-11T22:15:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:54 smithi110 ceph-mon[34830]: pgmap v403: 129 pgs: 129 active+clean; 90 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 264 op/s 2024-09-11T22:15:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:54 smithi092 ceph-mon[31173]: pgmap v403: 129 pgs: 129 active+clean; 90 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 264 op/s 2024-09-11T22:15:55.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:54 smithi153 ceph-mon[35045]: pgmap v403: 129 pgs: 129 active+clean; 90 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 264 op/s 2024-09-11T22:15:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:56 smithi110 ceph-mon[34830]: pgmap v404: 129 pgs: 129 active+clean; 99 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 224 op/s 2024-09-11T22:15:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:56 smithi092 ceph-mon[31173]: pgmap v404: 129 pgs: 129 active+clean; 99 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 224 op/s 2024-09-11T22:15:57.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:56 smithi153 ceph-mon[35045]: pgmap v404: 129 pgs: 129 active+clean; 99 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 224 op/s 2024-09-11T22:15:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:15:58 smithi110 ceph-mon[34830]: pgmap v405: 129 pgs: 129 active+clean; 103 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 216 op/s 2024-09-11T22:15:59.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:15:58 smithi092 ceph-mon[31173]: pgmap v405: 129 pgs: 129 active+clean; 103 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 216 op/s 2024-09-11T22:15:59.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:15:58 smithi153 ceph-mon[35045]: pgmap v405: 129 pgs: 129 active+clean; 103 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 216 op/s 2024-09-11T22:16:00.612 INFO:tasks.workunit.client.0.smithi092.stdout: 32 15099 304 9642 485 1262 62 2024-09-11T22:16:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:00 smithi110 ceph-mon[34830]: pgmap v406: 129 pgs: 129 active+clean; 110 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2024-09-11T22:16:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:00 smithi092 ceph-mon[31173]: pgmap v406: 129 pgs: 129 active+clean; 110 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2024-09-11T22:16:01.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:00 smithi153 ceph-mon[35045]: pgmap v406: 129 pgs: 129 active+clean; 110 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2024-09-11T22:16:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:02 smithi110 ceph-mon[34830]: pgmap v407: 129 pgs: 129 active+clean; 110 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 3.3 MiB/s wr, 177 op/s 2024-09-11T22:16:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:02 smithi092 ceph-mon[31173]: pgmap v407: 129 pgs: 129 active+clean; 110 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 3.3 MiB/s wr, 177 op/s 2024-09-11T22:16:03.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:02 smithi153 ceph-mon[35045]: pgmap v407: 129 pgs: 129 active+clean; 110 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 3.3 MiB/s wr, 177 op/s 2024-09-11T22:16:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:04 smithi110 ceph-mon[34830]: pgmap v408: 129 pgs: 129 active+clean; 119 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-09-11T22:16:05.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:04 smithi092 ceph-mon[31173]: pgmap v408: 129 pgs: 129 active+clean; 119 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-09-11T22:16:05.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:04 smithi153 ceph-mon[35045]: pgmap v408: 129 pgs: 129 active+clean; 119 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-09-11T22:16:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:05 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:06.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:05 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:06.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:06.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:05 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:07.220 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:06 smithi110 ceph-mon[34830]: pgmap v409: 129 pgs: 129 active+clean; 124 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 3.0 MiB/s wr, 181 op/s 2024-09-11T22:16:07.220 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:07.220 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:07.220 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:06 smithi092 ceph-mon[31173]: pgmap v409: 129 pgs: 129 active+clean; 124 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 3.0 MiB/s wr, 181 op/s 2024-09-11T22:16:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:07.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:07.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:06 smithi153 ceph-mon[35045]: pgmap v409: 129 pgs: 129 active+clean; 124 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 3.0 MiB/s wr, 181 op/s 2024-09-11T22:16:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:16:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:08 smithi110 ceph-mon[34830]: pgmap v410: 129 pgs: 129 active+clean; 125 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-11T22:16:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:08 smithi092 ceph-mon[31173]: pgmap v410: 129 pgs: 129 active+clean; 125 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-11T22:16:09.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:08 smithi153 ceph-mon[35045]: pgmap v410: 129 pgs: 129 active+clean; 125 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-11T22:16:10.612 INFO:tasks.workunit.client.0.smithi092.stdout: 37 14358 312 8937 233 1154 93 2024-09-11T22:16:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:10 smithi110 ceph-mon[34830]: pgmap v411: 129 pgs: 129 active+clean; 131 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 2.5 MiB/s wr, 172 op/s 2024-09-11T22:16:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:10 smithi092 ceph-mon[31173]: pgmap v411: 129 pgs: 129 active+clean; 131 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 2.5 MiB/s wr, 172 op/s 2024-09-11T22:16:11.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:10 smithi153 ceph-mon[35045]: pgmap v411: 129 pgs: 129 active+clean; 131 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 2.5 MiB/s wr, 172 op/s 2024-09-11T22:16:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:12 smithi110 ceph-mon[34830]: pgmap v412: 129 pgs: 129 active+clean; 134 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-09-11T22:16:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:12 smithi092 ceph-mon[31173]: pgmap v412: 129 pgs: 129 active+clean; 134 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-09-11T22:16:13.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:12 smithi153 ceph-mon[35045]: pgmap v412: 129 pgs: 129 active+clean; 134 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-09-11T22:16:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:14 smithi110 ceph-mon[34830]: pgmap v413: 129 pgs: 129 active+clean; 140 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-09-11T22:16:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:14 smithi092 ceph-mon[31173]: pgmap v413: 129 pgs: 129 active+clean; 140 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-09-11T22:16:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:14 smithi153 ceph-mon[35045]: pgmap v413: 129 pgs: 129 active+clean; 140 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-09-11T22:16:17.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:16 smithi110 ceph-mon[34830]: pgmap v414: 129 pgs: 129 active+clean; 142 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-11T22:16:17.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:16 smithi092 ceph-mon[31173]: pgmap v414: 129 pgs: 129 active+clean; 142 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-11T22:16:17.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:16 smithi153 ceph-mon[35045]: pgmap v414: 129 pgs: 129 active+clean; 142 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-11T22:16:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:18 smithi110 ceph-mon[34830]: pgmap v415: 129 pgs: 129 active+clean; 143 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-09-11T22:16:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:18 smithi092 ceph-mon[31173]: pgmap v415: 129 pgs: 129 active+clean; 143 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-09-11T22:16:19.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:18 smithi153 ceph-mon[35045]: pgmap v415: 129 pgs: 129 active+clean; 143 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-09-11T22:16:20.613 INFO:tasks.workunit.client.0.smithi092.stdout: 39 14455 161 8648 229 1456 161 2024-09-11T22:16:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:20 smithi110 ceph-mon[34830]: pgmap v416: 129 pgs: 129 active+clean; 150 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-09-11T22:16:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:20 smithi092 ceph-mon[31173]: pgmap v416: 129 pgs: 129 active+clean; 150 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-09-11T22:16:21.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:20 smithi153 ceph-mon[35045]: pgmap v416: 129 pgs: 129 active+clean; 150 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-09-11T22:16:22.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:21 smithi110 ceph-mon[34830]: pgmap v417: 129 pgs: 129 active+clean; 151 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-11T22:16:22.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:21 smithi092 ceph-mon[31173]: pgmap v417: 129 pgs: 129 active+clean; 151 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-11T22:16:22.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:21 smithi153 ceph-mon[35045]: pgmap v417: 129 pgs: 129 active+clean; 151 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-11T22:16:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:24 smithi110 ceph-mon[34830]: pgmap v418: 129 pgs: 129 active+clean; 158 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-09-11T22:16:25.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:24 smithi092 ceph-mon[31173]: pgmap v418: 129 pgs: 129 active+clean; 158 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-09-11T22:16:25.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:24 smithi153 ceph-mon[35045]: pgmap v418: 129 pgs: 129 active+clean; 158 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-09-11T22:16:27.070 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:26 smithi092 ceph-mon[31173]: pgmap v419: 129 pgs: 129 active+clean; 160 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-09-11T22:16:27.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:26 smithi110 ceph-mon[34830]: pgmap v419: 129 pgs: 129 active+clean; 160 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-09-11T22:16:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:26 smithi153 ceph-mon[35045]: pgmap v419: 129 pgs: 129 active+clean; 160 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-09-11T22:16:29.090 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:28 smithi153 ceph-mon[35045]: pgmap v420: 129 pgs: 129 active+clean; 162 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.9 MiB/s wr, 242 op/s 2024-09-11T22:16:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:28 smithi110 ceph-mon[34830]: pgmap v420: 129 pgs: 129 active+clean; 162 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.9 MiB/s wr, 242 op/s 2024-09-11T22:16:29.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:28 smithi092 ceph-mon[31173]: pgmap v420: 129 pgs: 129 active+clean; 162 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.9 MiB/s wr, 242 op/s 2024-09-11T22:16:30.613 INFO:tasks.workunit.client.0.smithi092.stdout: 42 15088 149 9589 168 1006 150 2024-09-11T22:16:31.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:30 smithi110 ceph-mon[34830]: pgmap v421: 129 pgs: 129 active+clean; 167 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-09-11T22:16:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:30 smithi092 ceph-mon[31173]: pgmap v421: 129 pgs: 129 active+clean; 167 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-09-11T22:16:31.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:30 smithi153 ceph-mon[35045]: pgmap v421: 129 pgs: 129 active+clean; 167 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-09-11T22:16:32.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:32 smithi110 ceph-mon[34830]: pgmap v422: 129 pgs: 129 active+clean; 167 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-11T22:16:32.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:32 smithi092 ceph-mon[31173]: pgmap v422: 129 pgs: 129 active+clean; 167 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-11T22:16:32.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:32 smithi153 ceph-mon[35045]: pgmap v422: 129 pgs: 129 active+clean; 167 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-11T22:16:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:34 smithi110 ceph-mon[34830]: pgmap v423: 129 pgs: 129 active+clean; 174 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-09-11T22:16:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:34 smithi092 ceph-mon[31173]: pgmap v423: 129 pgs: 129 active+clean; 174 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-09-11T22:16:35.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:34 smithi153 ceph-mon[35045]: pgmap v423: 129 pgs: 129 active+clean; 174 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-09-11T22:16:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:36 smithi110 ceph-mon[34830]: pgmap v424: 129 pgs: 129 active+clean; 175 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-11T22:16:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:36 smithi092 ceph-mon[31173]: pgmap v424: 129 pgs: 129 active+clean; 175 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-11T22:16:37.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:36 smithi153 ceph-mon[35045]: pgmap v424: 129 pgs: 129 active+clean; 175 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-11T22:16:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:38 smithi110 ceph-mon[34830]: pgmap v425: 129 pgs: 129 active+clean; 177 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-11T22:16:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:38 smithi092 ceph-mon[31173]: pgmap v425: 129 pgs: 129 active+clean; 177 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-11T22:16:39.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:38 smithi153 ceph-mon[35045]: pgmap v425: 129 pgs: 129 active+clean; 177 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-11T22:16:40.613 INFO:tasks.workunit.client.0.smithi092.stdout: 44 14521 179 9078 148 1624 125 2024-09-11T22:16:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:40 smithi110 ceph-mon[34830]: pgmap v426: 129 pgs: 129 active+clean; 184 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-09-11T22:16:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:40 smithi092 ceph-mon[31173]: pgmap v426: 129 pgs: 129 active+clean; 184 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-09-11T22:16:41.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:40 smithi153 ceph-mon[35045]: pgmap v426: 129 pgs: 129 active+clean; 184 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-09-11T22:16:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:42 smithi110 ceph-mon[34830]: pgmap v427: 129 pgs: 129 active+clean; 184 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-09-11T22:16:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:42 smithi092 ceph-mon[31173]: pgmap v427: 129 pgs: 129 active+clean; 184 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-09-11T22:16:43.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:42 smithi153 ceph-mon[35045]: pgmap v427: 129 pgs: 129 active+clean; 184 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-09-11T22:16:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:44 smithi110 ceph-mon[34830]: pgmap v428: 129 pgs: 129 active+clean; 189 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-11T22:16:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:44 smithi092 ceph-mon[31173]: pgmap v428: 129 pgs: 129 active+clean; 189 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-11T22:16:45.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:44 smithi153 ceph-mon[35045]: pgmap v428: 129 pgs: 129 active+clean; 189 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-11T22:16:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:46 smithi110 ceph-mon[34830]: pgmap v429: 129 pgs: 129 active+clean; 191 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-11T22:16:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:46 smithi092 ceph-mon[31173]: pgmap v429: 129 pgs: 129 active+clean; 191 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-11T22:16:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:46 smithi153 ceph-mon[35045]: pgmap v429: 129 pgs: 129 active+clean; 191 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-11T22:16:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:48 smithi110 ceph-mon[34830]: pgmap v430: 129 pgs: 129 active+clean; 192 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-11T22:16:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:48 smithi092 ceph-mon[31173]: pgmap v430: 129 pgs: 129 active+clean; 192 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-11T22:16:49.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:48 smithi153 ceph-mon[35045]: pgmap v430: 129 pgs: 129 active+clean; 192 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-11T22:16:50.613 INFO:tasks.workunit.client.0.smithi092.stdout: 45 14919 57 9618 94 1343 146 2024-09-11T22:16:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:50 smithi110 ceph-mon[34830]: pgmap v431: 129 pgs: 129 active+clean; 195 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-09-11T22:16:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:50 smithi092 ceph-mon[31173]: pgmap v431: 129 pgs: 129 active+clean; 195 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-09-11T22:16:51.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:50 smithi153 ceph-mon[35045]: pgmap v431: 129 pgs: 129 active+clean; 195 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-09-11T22:16:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:52 smithi110 ceph-mon[34830]: pgmap v432: 129 pgs: 129 active+clean; 197 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-09-11T22:16:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:52 smithi092 ceph-mon[31173]: pgmap v432: 129 pgs: 129 active+clean; 197 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-09-11T22:16:53.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:52 smithi153 ceph-mon[35045]: pgmap v432: 129 pgs: 129 active+clean; 197 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-09-11T22:16:55.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:54 smithi110 ceph-mon[34830]: pgmap v433: 129 pgs: 129 active+clean; 200 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 249 op/s 2024-09-11T22:16:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:54 smithi092 ceph-mon[31173]: pgmap v433: 129 pgs: 129 active+clean; 200 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 249 op/s 2024-09-11T22:16:55.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:54 smithi153 ceph-mon[35045]: pgmap v433: 129 pgs: 129 active+clean; 200 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 249 op/s 2024-09-11T22:16:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:56 smithi110 ceph-mon[34830]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 200 op/s 2024-09-11T22:16:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:56 smithi092 ceph-mon[31173]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 200 op/s 2024-09-11T22:16:57.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:56 smithi153 ceph-mon[35045]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 200 op/s 2024-09-11T22:16:59.090 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:16:58 smithi153 ceph-mon[35045]: pgmap v435: 129 pgs: 129 active+clean; 205 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-09-11T22:16:59.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:16:58 smithi110 ceph-mon[34830]: pgmap v435: 129 pgs: 129 active+clean; 205 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-09-11T22:16:59.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:16:58 smithi092 ceph-mon[31173]: pgmap v435: 129 pgs: 129 active+clean; 205 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-09-11T22:17:00.614 INFO:tasks.workunit.client.0.smithi092.stdout: 47 14455 115 9958 74 2197 117 2024-09-11T22:17:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:00 smithi110 ceph-mon[34830]: pgmap v436: 129 pgs: 129 active+clean; 207 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-11T22:17:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:00 smithi092 ceph-mon[31173]: pgmap v436: 129 pgs: 129 active+clean; 207 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-11T22:17:01.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:00 smithi153 ceph-mon[35045]: pgmap v436: 129 pgs: 129 active+clean; 207 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-11T22:17:03.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:02 smithi110 ceph-mon[34830]: pgmap v437: 129 pgs: 129 active+clean; 208 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-09-11T22:17:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:02 smithi092 ceph-mon[31173]: pgmap v437: 129 pgs: 129 active+clean; 208 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-09-11T22:17:03.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:02 smithi153 ceph-mon[35045]: pgmap v437: 129 pgs: 129 active+clean; 208 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-09-11T22:17:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:04 smithi110 ceph-mon[34830]: pgmap v438: 129 pgs: 129 active+clean; 213 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-09-11T22:17:05.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:04 smithi092 ceph-mon[31173]: pgmap v438: 129 pgs: 129 active+clean; 213 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-09-11T22:17:05.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:04 smithi153 ceph-mon[35045]: pgmap v438: 129 pgs: 129 active+clean; 213 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-09-11T22:17:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:06 smithi110 ceph-mon[34830]: pgmap v439: 129 pgs: 129 active+clean; 216 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:17:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:17:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:06 smithi092 ceph-mon[31173]: pgmap v439: 129 pgs: 129 active+clean; 216 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:17:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:07.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:17:07.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:06 smithi153 ceph-mon[35045]: pgmap v439: 129 pgs: 129 active+clean; 216 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:17:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:17:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:08 smithi110 ceph-mon[34830]: pgmap v440: 129 pgs: 129 active+clean; 216 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-11T22:17:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:08 smithi092 ceph-mon[31173]: pgmap v440: 129 pgs: 129 active+clean; 216 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-11T22:17:09.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:08 smithi153 ceph-mon[35045]: pgmap v440: 129 pgs: 129 active+clean; 216 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-11T22:17:10.617 INFO:tasks.workunit.client.0.smithi092.stdout: 49 14052 90 9706 98 1806 95 2024-09-11T22:17:11.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:10 smithi110 ceph-mon[34830]: pgmap v441: 129 pgs: 129 active+clean; 222 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-09-11T22:17:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:10 smithi092 ceph-mon[31173]: pgmap v441: 129 pgs: 129 active+clean; 222 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-09-11T22:17:11.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:10 smithi153 ceph-mon[35045]: pgmap v441: 129 pgs: 129 active+clean; 222 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-09-11T22:17:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:12 smithi110 ceph-mon[34830]: pgmap v442: 129 pgs: 129 active+clean; 222 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-09-11T22:17:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:12 smithi092 ceph-mon[31173]: pgmap v442: 129 pgs: 129 active+clean; 222 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-09-11T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:12 smithi153 ceph-mon[35045]: pgmap v442: 129 pgs: 129 active+clean; 222 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-09-11T22:17:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:14 smithi110 ceph-mon[34830]: pgmap v443: 129 pgs: 129 active+clean; 225 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-09-11T22:17:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:14 smithi092 ceph-mon[31173]: pgmap v443: 129 pgs: 129 active+clean; 225 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-09-11T22:17:15.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:14 smithi153 ceph-mon[35045]: pgmap v443: 129 pgs: 129 active+clean; 225 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-09-11T22:17:17.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:16 smithi110 ceph-mon[34830]: pgmap v444: 129 pgs: 129 active+clean; 229 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-09-11T22:17:17.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:16 smithi092 ceph-mon[31173]: pgmap v444: 129 pgs: 129 active+clean; 229 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-09-11T22:17:17.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:16 smithi153 ceph-mon[35045]: pgmap v444: 129 pgs: 129 active+clean; 229 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-09-11T22:17:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:18 smithi110 ceph-mon[34830]: pgmap v445: 129 pgs: 129 active+clean; 229 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-09-11T22:17:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:18 smithi092 ceph-mon[31173]: pgmap v445: 129 pgs: 129 active+clean; 229 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-09-11T22:17:19.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:18 smithi153 ceph-mon[35045]: pgmap v445: 129 pgs: 129 active+clean; 229 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-09-11T22:17:20.613 INFO:tasks.workunit.client.0.smithi092.stdout: 51 13341 93 9633 94 2882 84 2024-09-11T22:17:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:20 smithi110 ceph-mon[34830]: pgmap v446: 129 pgs: 129 active+clean; 235 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-11T22:17:21.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:20 smithi092 ceph-mon[31173]: pgmap v446: 129 pgs: 129 active+clean; 235 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-11T22:17:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:20 smithi153 ceph-mon[35045]: pgmap v446: 129 pgs: 129 active+clean; 235 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-11T22:17:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:22 smithi110 ceph-mon[34830]: pgmap v447: 129 pgs: 129 active+clean; 235 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-11T22:17:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:22 smithi092 ceph-mon[31173]: pgmap v447: 129 pgs: 129 active+clean; 235 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-11T22:17:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:22 smithi153 ceph-mon[35045]: pgmap v447: 129 pgs: 129 active+clean; 235 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-11T22:17:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:24 smithi110 ceph-mon[34830]: pgmap v448: 129 pgs: 129 active+clean; 238 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-09-11T22:17:25.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:24 smithi092 ceph-mon[31173]: pgmap v448: 129 pgs: 129 active+clean; 238 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-09-11T22:17:25.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:24 smithi153 ceph-mon[35045]: pgmap v448: 129 pgs: 129 active+clean; 238 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-09-11T22:17:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:26 smithi110 ceph-mon[34830]: pgmap v449: 129 pgs: 129 active+clean; 241 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:17:27.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:26 smithi092 ceph-mon[31173]: pgmap v449: 129 pgs: 129 active+clean; 241 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:17:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:26 smithi153 ceph-mon[35045]: pgmap v449: 129 pgs: 129 active+clean; 241 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-09-11T22:17:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:28 smithi110 ceph-mon[34830]: pgmap v450: 129 pgs: 129 active+clean; 241 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-11T22:17:29.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:28 smithi092 ceph-mon[31173]: pgmap v450: 129 pgs: 129 active+clean; 241 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-11T22:17:29.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:28 smithi153 ceph-mon[35045]: pgmap v450: 129 pgs: 129 active+clean; 241 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-11T22:17:30.613 INFO:tasks.workunit.client.0.smithi092.stdout: 53 13460 163 8776 87 2634 113 2024-09-11T22:17:31.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:30 smithi110 ceph-mon[34830]: pgmap v451: 129 pgs: 129 active+clean; 250 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-11T22:17:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:30 smithi092 ceph-mon[31173]: pgmap v451: 129 pgs: 129 active+clean; 250 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-11T22:17:31.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:30 smithi153 ceph-mon[35045]: pgmap v451: 129 pgs: 129 active+clean; 250 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-11T22:17:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:32 smithi110 ceph-mon[34830]: pgmap v452: 129 pgs: 129 active+clean; 250 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-11T22:17:33.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:32 smithi092 ceph-mon[31173]: pgmap v452: 129 pgs: 129 active+clean; 250 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-11T22:17:33.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:32 smithi153 ceph-mon[35045]: pgmap v452: 129 pgs: 129 active+clean; 250 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-11T22:17:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:34 smithi110 ceph-mon[34830]: pgmap v453: 129 pgs: 129 active+clean; 254 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-11T22:17:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:34 smithi092 ceph-mon[31173]: pgmap v453: 129 pgs: 129 active+clean; 254 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-11T22:17:35.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:34 smithi153 ceph-mon[35045]: pgmap v453: 129 pgs: 129 active+clean; 254 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-11T22:17:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:36 smithi110 ceph-mon[34830]: pgmap v454: 129 pgs: 129 active+clean; 259 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:17:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:36 smithi092 ceph-mon[31173]: pgmap v454: 129 pgs: 129 active+clean; 259 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:17:37.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:36 smithi153 ceph-mon[35045]: pgmap v454: 129 pgs: 129 active+clean; 259 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:17:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:38 smithi110 ceph-mon[34830]: pgmap v455: 129 pgs: 129 active+clean; 260 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-11T22:17:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:38 smithi092 ceph-mon[31173]: pgmap v455: 129 pgs: 129 active+clean; 260 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-11T22:17:39.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:38 smithi153 ceph-mon[35045]: pgmap v455: 129 pgs: 129 active+clean; 260 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-11T22:17:40.613 INFO:tasks.workunit.client.0.smithi092.stdout: 55 13673 133 9768 93 2385 90 2024-09-11T22:17:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:40 smithi110 ceph-mon[34830]: pgmap v456: 129 pgs: 129 active+clean; 266 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-09-11T22:17:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:40 smithi092 ceph-mon[31173]: pgmap v456: 129 pgs: 129 active+clean; 266 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-09-11T22:17:41.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:40 smithi153 ceph-mon[35045]: pgmap v456: 129 pgs: 129 active+clean; 266 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-09-11T22:17:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:42 smithi110 ceph-mon[34830]: pgmap v457: 129 pgs: 129 active+clean; 266 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-09-11T22:17:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:42 smithi092 ceph-mon[31173]: pgmap v457: 129 pgs: 129 active+clean; 266 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-09-11T22:17:43.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:42 smithi153 ceph-mon[35045]: pgmap v457: 129 pgs: 129 active+clean; 266 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-09-11T22:17:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:44 smithi110 ceph-mon[34830]: pgmap v458: 129 pgs: 129 active+clean; 272 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-09-11T22:17:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:44 smithi092 ceph-mon[31173]: pgmap v458: 129 pgs: 129 active+clean; 272 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-09-11T22:17:45.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:44 smithi153 ceph-mon[35045]: pgmap v458: 129 pgs: 129 active+clean; 272 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-09-11T22:17:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:46 smithi110 ceph-mon[34830]: pgmap v459: 129 pgs: 129 active+clean; 275 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T22:17:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:46 smithi092 ceph-mon[31173]: pgmap v459: 129 pgs: 129 active+clean; 275 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T22:17:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:46 smithi153 ceph-mon[35045]: pgmap v459: 129 pgs: 129 active+clean; 275 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T22:17:49.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:48 smithi110 ceph-mon[34830]: pgmap v460: 129 pgs: 129 active+clean; 275 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-11T22:17:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:48 smithi092 ceph-mon[31173]: pgmap v460: 129 pgs: 129 active+clean; 275 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-11T22:17:49.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:48 smithi153 ceph-mon[35045]: pgmap v460: 129 pgs: 129 active+clean; 275 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-11T22:17:50.613 INFO:tasks.workunit.client.0.smithi092.stdout: 56 13901 86 9408 91 2294 118 2024-09-11T22:17:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:50 smithi110 ceph-mon[34830]: pgmap v461: 129 pgs: 129 active+clean; 281 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-11T22:17:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:50 smithi092 ceph-mon[31173]: pgmap v461: 129 pgs: 129 active+clean; 281 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-11T22:17:51.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:50 smithi153 ceph-mon[35045]: pgmap v461: 129 pgs: 129 active+clean; 281 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-11T22:17:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:52 smithi110 ceph-mon[34830]: pgmap v462: 129 pgs: 129 active+clean; 283 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-09-11T22:17:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:52 smithi092 ceph-mon[31173]: pgmap v462: 129 pgs: 129 active+clean; 283 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-09-11T22:17:53.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:52 smithi153 ceph-mon[35045]: pgmap v462: 129 pgs: 129 active+clean; 283 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-09-11T22:17:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:54 smithi110 ceph-mon[34830]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-11T22:17:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:54 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:17:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:54 smithi092 ceph-mon[31173]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-11T22:17:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:54 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:17:55.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:54 smithi153 ceph-mon[35045]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-11T22:17:55.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:54 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:17:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:56 smithi110 ceph-mon[34830]: pgmap v464: 129 pgs: 129 active+clean; 290 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-11T22:17:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:56 smithi092 ceph-mon[31173]: pgmap v464: 129 pgs: 129 active+clean; 290 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-11T22:17:57.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:56 smithi153 ceph-mon[35045]: pgmap v464: 129 pgs: 129 active+clean; 290 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-11T22:17:58.221 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:17:57 smithi110 ceph-mon[34830]: pgmap v465: 129 pgs: 129 active+clean; 291 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T22:17:58.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:17:57 smithi092 ceph-mon[31173]: pgmap v465: 129 pgs: 129 active+clean; 291 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T22:17:58.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:17:57 smithi153 ceph-mon[35045]: pgmap v465: 129 pgs: 129 active+clean; 291 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T22:18:00.613 INFO:tasks.workunit.client.0.smithi092.stdout: 56 13169 71 9491 70 3190 126 2024-09-11T22:18:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:00 smithi110 ceph-mon[34830]: pgmap v466: 129 pgs: 129 active+clean; 297 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-11T22:18:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:00 smithi092 ceph-mon[31173]: pgmap v466: 129 pgs: 129 active+clean; 297 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-11T22:18:01.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:00 smithi153 ceph-mon[35045]: pgmap v466: 129 pgs: 129 active+clean; 297 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-11T22:18:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:02 smithi110 ceph-mon[34830]: pgmap v467: 129 pgs: 129 active+clean; 297 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-11T22:18:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:02 smithi092 ceph-mon[31173]: pgmap v467: 129 pgs: 129 active+clean; 297 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-11T22:18:03.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:02 smithi153 ceph-mon[35045]: pgmap v467: 129 pgs: 129 active+clean; 297 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-11T22:18:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:04 smithi110 ceph-mon[34830]: pgmap v468: 129 pgs: 129 active+clean; 301 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-09-11T22:18:05.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:04 smithi092 ceph-mon[31173]: pgmap v468: 129 pgs: 129 active+clean; 301 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-09-11T22:18:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:04 smithi153 ceph-mon[35045]: pgmap v468: 129 pgs: 129 active+clean; 301 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-09-11T22:18:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:06 smithi110 ceph-mon[34830]: pgmap v469: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-11T22:18:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:06 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:06 smithi092 ceph-mon[31173]: pgmap v469: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-11T22:18:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:06 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:07.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:06 smithi153 ceph-mon[35045]: pgmap v469: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-11T22:18:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:06 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:08.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:08.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:08.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:18:08.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:08.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:08.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:18:08.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:08.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:08.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:18:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:08 smithi110 ceph-mon[34830]: pgmap v470: 129 pgs: 129 active+clean; 307 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T22:18:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:08 smithi092 ceph-mon[31173]: pgmap v470: 129 pgs: 129 active+clean; 307 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T22:18:09.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:08 smithi153 ceph-mon[35045]: pgmap v470: 129 pgs: 129 active+clean; 307 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T22:18:10.613 INFO:tasks.workunit.client.0.smithi092.stdout: 57 13245 24 8761 121 2294 115 2024-09-11T22:18:11.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:10 smithi110 ceph-mon[34830]: pgmap v471: 129 pgs: 129 active+clean; 313 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2024-09-11T22:18:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:10 smithi092 ceph-mon[31173]: pgmap v471: 129 pgs: 129 active+clean; 313 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2024-09-11T22:18:11.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:10 smithi153 ceph-mon[35045]: pgmap v471: 129 pgs: 129 active+clean; 313 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2024-09-11T22:18:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:12 smithi110 ceph-mon[34830]: pgmap v472: 129 pgs: 129 active+clean; 313 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-11T22:18:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:12 smithi092 ceph-mon[31173]: pgmap v472: 129 pgs: 129 active+clean; 313 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-11T22:18:13.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:12 smithi153 ceph-mon[35045]: pgmap v472: 129 pgs: 129 active+clean; 313 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-11T22:18:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:14 smithi110 ceph-mon[34830]: pgmap v473: 129 pgs: 129 active+clean; 318 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-11T22:18:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:14 smithi092 ceph-mon[31173]: pgmap v473: 129 pgs: 129 active+clean; 318 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-11T22:18:15.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:14 smithi153 ceph-mon[35045]: pgmap v473: 129 pgs: 129 active+clean; 318 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-11T22:18:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:16 smithi110 ceph-mon[34830]: pgmap v474: 129 pgs: 129 active+clean; 320 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-09-11T22:18:17.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:16 smithi092 ceph-mon[31173]: pgmap v474: 129 pgs: 129 active+clean; 320 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-09-11T22:18:17.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:16 smithi153 ceph-mon[35045]: pgmap v474: 129 pgs: 129 active+clean; 320 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-09-11T22:18:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:18 smithi110 ceph-mon[34830]: pgmap v475: 129 pgs: 129 active+clean; 322 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-09-11T22:18:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:18 smithi092 ceph-mon[31173]: pgmap v475: 129 pgs: 129 active+clean; 322 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-09-11T22:18:19.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:18 smithi153 ceph-mon[35045]: pgmap v475: 129 pgs: 129 active+clean; 322 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-09-11T22:18:20.613 INFO:tasks.workunit.client.0.smithi092.stdout: 58 12992 34 9065 78 3716 102 2024-09-11T22:18:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:20 smithi110 ceph-mon[34830]: pgmap v476: 129 pgs: 129 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-09-11T22:18:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:20 smithi092 ceph-mon[31173]: pgmap v476: 129 pgs: 129 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-09-11T22:18:21.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:20 smithi153 ceph-mon[35045]: pgmap v476: 129 pgs: 129 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-09-11T22:18:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:22 smithi110 ceph-mon[34830]: pgmap v477: 129 pgs: 129 active+clean; 329 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-09-11T22:18:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:22 smithi092 ceph-mon[31173]: pgmap v477: 129 pgs: 129 active+clean; 329 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-09-11T22:18:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:22 smithi153 ceph-mon[35045]: pgmap v477: 129 pgs: 129 active+clean; 329 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-09-11T22:18:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:24 smithi110 ceph-mon[34830]: pgmap v478: 129 pgs: 129 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-11T22:18:25.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:24 smithi092 ceph-mon[31173]: pgmap v478: 129 pgs: 129 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-11T22:18:25.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:24 smithi153 ceph-mon[35045]: pgmap v478: 129 pgs: 129 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-11T22:18:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:26 smithi110 ceph-mon[34830]: pgmap v479: 129 pgs: 129 active+clean; 337 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-11T22:18:27.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:26 smithi092 ceph-mon[31173]: pgmap v479: 129 pgs: 129 active+clean; 337 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-11T22:18:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:26 smithi153 ceph-mon[35045]: pgmap v479: 129 pgs: 129 active+clean; 337 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-11T22:18:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:28 smithi110 ceph-mon[34830]: pgmap v480: 129 pgs: 129 active+clean; 337 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-11T22:18:30.695 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:28 smithi153 ceph-mon[35045]: pgmap v480: 129 pgs: 129 active+clean; 337 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-11T22:18:30.695 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:28 smithi092 ceph-mon[31173]: pgmap v480: 129 pgs: 129 active+clean; 337 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-11T22:18:30.695 INFO:tasks.workunit.client.0.smithi092.stdout: 61 13142 117 9318 52 3565 121 2024-09-11T22:18:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:30 smithi110 ceph-mon[34830]: pgmap v481: 129 pgs: 129 active+clean; 342 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-09-11T22:18:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:30 smithi092 ceph-mon[31173]: pgmap v481: 129 pgs: 129 active+clean; 342 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-09-11T22:18:31.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:30 smithi153 ceph-mon[35045]: pgmap v481: 129 pgs: 129 active+clean; 342 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-09-11T22:18:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:32 smithi110 ceph-mon[34830]: pgmap v482: 129 pgs: 129 active+clean; 344 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-11T22:18:33.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:32 smithi092 ceph-mon[31173]: pgmap v482: 129 pgs: 129 active+clean; 344 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-11T22:18:33.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:32 smithi153 ceph-mon[35045]: pgmap v482: 129 pgs: 129 active+clean; 344 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-11T22:18:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:34 smithi110 ceph-mon[34830]: pgmap v483: 129 pgs: 129 active+clean; 347 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-09-11T22:18:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:34 smithi092 ceph-mon[31173]: pgmap v483: 129 pgs: 129 active+clean; 347 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-09-11T22:18:35.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:34 smithi153 ceph-mon[35045]: pgmap v483: 129 pgs: 129 active+clean; 347 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-09-11T22:18:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:36 smithi110 ceph-mon[34830]: pgmap v484: 129 pgs: 129 active+clean; 352 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-09-11T22:18:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:36 smithi092 ceph-mon[31173]: pgmap v484: 129 pgs: 129 active+clean; 352 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-09-11T22:18:37.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:36 smithi153 ceph-mon[35045]: pgmap v484: 129 pgs: 129 active+clean; 352 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-09-11T22:18:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:38 smithi110 ceph-mon[34830]: pgmap v485: 129 pgs: 129 active+clean; 353 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-11T22:18:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:38 smithi092 ceph-mon[31173]: pgmap v485: 129 pgs: 129 active+clean; 353 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-11T22:18:39.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:38 smithi153 ceph-mon[35045]: pgmap v485: 129 pgs: 129 active+clean; 353 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-11T22:18:40.614 INFO:tasks.workunit.client.0.smithi092.stdout: 61 13212 81 9340 46 3393 122 2024-09-11T22:18:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:40 smithi110 ceph-mon[34830]: pgmap v486: 129 pgs: 129 active+clean; 359 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-11T22:18:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:40 smithi092 ceph-mon[31173]: pgmap v486: 129 pgs: 129 active+clean; 359 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-11T22:18:41.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:40 smithi153 ceph-mon[35045]: pgmap v486: 129 pgs: 129 active+clean; 359 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-11T22:18:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:42 smithi110 ceph-mon[34830]: pgmap v487: 129 pgs: 129 active+clean; 360 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-09-11T22:18:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:42 smithi092 ceph-mon[31173]: pgmap v487: 129 pgs: 129 active+clean; 360 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-09-11T22:18:43.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:42 smithi153 ceph-mon[35045]: pgmap v487: 129 pgs: 129 active+clean; 360 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-09-11T22:18:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:44 smithi110 ceph-mon[34830]: pgmap v488: 129 pgs: 129 active+clean; 362 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-09-11T22:18:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:44 smithi092 ceph-mon[31173]: pgmap v488: 129 pgs: 129 active+clean; 362 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-09-11T22:18:45.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:44 smithi153 ceph-mon[35045]: pgmap v488: 129 pgs: 129 active+clean; 362 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-09-11T22:18:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:46 smithi110 ceph-mon[34830]: pgmap v489: 129 pgs: 129 active+clean; 369 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-11T22:18:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:46 smithi092 ceph-mon[31173]: pgmap v489: 129 pgs: 129 active+clean; 369 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-11T22:18:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:46 smithi153 ceph-mon[35045]: pgmap v489: 129 pgs: 129 active+clean; 369 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-11T22:18:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:48 smithi110 ceph-mon[34830]: pgmap v490: 129 pgs: 129 active+clean; 371 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T22:18:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:48 smithi092 ceph-mon[31173]: pgmap v490: 129 pgs: 129 active+clean; 371 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T22:18:49.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:48 smithi153 ceph-mon[35045]: pgmap v490: 129 pgs: 129 active+clean; 371 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T22:18:50.613 INFO:tasks.workunit.client.0.smithi092.stdout: 62 12538 50 8897 92 3249 196 2024-09-11T22:18:51.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:50 smithi110 ceph-mon[34830]: pgmap v491: 129 pgs: 129 active+clean; 378 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-09-11T22:18:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:50 smithi092 ceph-mon[31173]: pgmap v491: 129 pgs: 129 active+clean; 378 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-09-11T22:18:51.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:50 smithi153 ceph-mon[35045]: pgmap v491: 129 pgs: 129 active+clean; 378 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-09-11T22:18:53.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:52 smithi110 ceph-mon[34830]: pgmap v492: 129 pgs: 129 active+clean; 378 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-09-11T22:18:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:52 smithi092 ceph-mon[31173]: pgmap v492: 129 pgs: 129 active+clean; 378 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-09-11T22:18:53.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:52 smithi153 ceph-mon[35045]: pgmap v492: 129 pgs: 129 active+clean; 378 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-09-11T22:18:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:54 smithi110 ceph-mon[34830]: pgmap v493: 129 pgs: 129 active+clean; 385 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-09-11T22:18:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:54 smithi092 ceph-mon[31173]: pgmap v493: 129 pgs: 129 active+clean; 385 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-09-11T22:18:55.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:54 smithi153 ceph-mon[35045]: pgmap v493: 129 pgs: 129 active+clean; 385 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-09-11T22:18:57.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:56 smithi110 ceph-mon[34830]: pgmap v494: 129 pgs: 129 active+clean; 389 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 273 op/s 2024-09-11T22:18:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:56 smithi092 ceph-mon[31173]: pgmap v494: 129 pgs: 129 active+clean; 389 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 273 op/s 2024-09-11T22:18:57.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:56 smithi153 ceph-mon[35045]: pgmap v494: 129 pgs: 129 active+clean; 389 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 273 op/s 2024-09-11T22:18:59.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:18:58 smithi110 ceph-mon[34830]: pgmap v495: 129 pgs: 129 active+clean; 390 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 218 op/s 2024-09-11T22:18:59.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:18:58 smithi092 ceph-mon[31173]: pgmap v495: 129 pgs: 129 active+clean; 390 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 218 op/s 2024-09-11T22:18:59.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:18:58 smithi153 ceph-mon[35045]: pgmap v495: 129 pgs: 129 active+clean; 390 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 218 op/s 2024-09-11T22:19:00.614 INFO:tasks.workunit.client.0.smithi092.stdout: 62 12632 111 8964 52 3227 150 2024-09-11T22:19:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:00 smithi110 ceph-mon[34830]: pgmap v496: 129 pgs: 129 active+clean; 397 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-11T22:19:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:00 smithi092 ceph-mon[31173]: pgmap v496: 129 pgs: 129 active+clean; 397 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-11T22:19:01.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:00 smithi153 ceph-mon[35045]: pgmap v496: 129 pgs: 129 active+clean; 397 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-11T22:19:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:02 smithi110 ceph-mon[34830]: pgmap v497: 129 pgs: 129 active+clean; 399 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T22:19:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:02 smithi092 ceph-mon[31173]: pgmap v497: 129 pgs: 129 active+clean; 399 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T22:19:03.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:02 smithi153 ceph-mon[35045]: pgmap v497: 129 pgs: 129 active+clean; 399 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T22:19:05.219 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:04 smithi110 ceph-mon[34830]: pgmap v498: 129 pgs: 129 active+clean; 404 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-11T22:19:05.284 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:04 smithi092 ceph-mon[31173]: pgmap v498: 129 pgs: 129 active+clean; 404 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-11T22:19:05.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:04 smithi153 ceph-mon[35045]: pgmap v498: 129 pgs: 129 active+clean; 404 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-11T22:19:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:06 smithi110 ceph-mon[34830]: pgmap v499: 129 pgs: 129 active+clean; 408 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-11T22:19:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:06 smithi092 ceph-mon[31173]: pgmap v499: 129 pgs: 129 active+clean; 408 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-11T22:19:07.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:06 smithi153 ceph-mon[35045]: pgmap v499: 129 pgs: 129 active+clean; 408 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-11T22:19:08.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:07 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:08.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:07 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:08.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:07 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:08 smithi110 ceph-mon[34830]: pgmap v500: 129 pgs: 129 active+clean; 408 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T22:19:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:19:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:08 smithi092 ceph-mon[31173]: pgmap v500: 129 pgs: 129 active+clean; 408 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T22:19:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:09.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:09.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:19:09.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:08 smithi153 ceph-mon[35045]: pgmap v500: 129 pgs: 129 active+clean; 408 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T22:19:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:19:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:09 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:19:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:09 smithi110 ceph-mon[34830]: pgmap v501: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2024-09-11T22:19:10.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:09 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:19:10.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:09 smithi092 ceph-mon[31173]: pgmap v501: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2024-09-11T22:19:10.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:09 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:19:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:09 smithi153 ceph-mon[35045]: pgmap v501: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2024-09-11T22:19:10.614 INFO:tasks.workunit.client.0.smithi092.stdout: 62 12475 19 9029 100 4276 114 2024-09-11T22:19:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:12 smithi110 ceph-mon[34830]: pgmap v502: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 150 op/s 2024-09-11T22:19:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:12 smithi092 ceph-mon[31173]: pgmap v502: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 150 op/s 2024-09-11T22:19:13.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:12 smithi153 ceph-mon[35045]: pgmap v502: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 150 op/s 2024-09-11T22:19:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:14 smithi110 ceph-mon[34830]: pgmap v503: 129 pgs: 129 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-09-11T22:19:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:14 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:19:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:14 smithi092 ceph-mon[31173]: pgmap v503: 129 pgs: 129 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-09-11T22:19:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:14 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:19:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:14 smithi153 ceph-mon[35045]: pgmap v503: 129 pgs: 129 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-09-11T22:19:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:14 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:19:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:16 smithi110 ceph-mon[34830]: pgmap v504: 129 pgs: 129 active+clean; 425 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-09-11T22:19:17.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:16 smithi092 ceph-mon[31173]: pgmap v504: 129 pgs: 129 active+clean; 425 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-09-11T22:19:17.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:16 smithi153 ceph-mon[35045]: pgmap v504: 129 pgs: 129 active+clean; 425 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-09-11T22:19:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:18 smithi110 ceph-mon[34830]: pgmap v505: 129 pgs: 129 active+clean; 426 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T22:19:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:18 smithi092 ceph-mon[31173]: pgmap v505: 129 pgs: 129 active+clean; 426 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T22:19:19.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:18 smithi153 ceph-mon[35045]: pgmap v505: 129 pgs: 129 active+clean; 426 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T22:19:20.614 INFO:tasks.workunit.client.0.smithi092.stdout: 64 13032 54 9336 49 3402 108 2024-09-11T22:19:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:20 smithi110 ceph-mon[34830]: pgmap v506: 129 pgs: 129 active+clean; 432 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-11T22:19:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:20 smithi092 ceph-mon[31173]: pgmap v506: 129 pgs: 129 active+clean; 432 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-11T22:19:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:20 smithi153 ceph-mon[35045]: pgmap v506: 129 pgs: 129 active+clean; 432 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-11T22:19:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:22 smithi110 ceph-mon[34830]: pgmap v507: 129 pgs: 129 active+clean; 432 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-11T22:19:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:22 smithi092 ceph-mon[31173]: pgmap v507: 129 pgs: 129 active+clean; 432 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-11T22:19:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:22 smithi153 ceph-mon[35045]: pgmap v507: 129 pgs: 129 active+clean; 432 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-11T22:19:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:24 smithi110 ceph-mon[34830]: pgmap v508: 129 pgs: 129 active+clean; 438 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T22:19:25.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:24 smithi092 ceph-mon[31173]: pgmap v508: 129 pgs: 129 active+clean; 438 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T22:19:25.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:24 smithi153 ceph-mon[35045]: pgmap v508: 129 pgs: 129 active+clean; 438 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T22:19:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:26 smithi110 ceph-mon[34830]: pgmap v509: 129 pgs: 129 active+clean; 440 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T22:19:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:26 smithi092 ceph-mon[31173]: pgmap v509: 129 pgs: 129 active+clean; 440 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T22:19:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:26 smithi153 ceph-mon[35045]: pgmap v509: 129 pgs: 129 active+clean; 440 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T22:19:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:28 smithi110 ceph-mon[34830]: pgmap v510: 129 pgs: 129 active+clean; 442 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:19:29.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:28 smithi092 ceph-mon[31173]: pgmap v510: 129 pgs: 129 active+clean; 442 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:19:29.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:28 smithi153 ceph-mon[35045]: pgmap v510: 129 pgs: 129 active+clean; 442 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:19:30.614 INFO:tasks.workunit.client.0.smithi092.stdout: 64 13279 73 9464 51 3224 153 2024-09-11T22:19:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:30 smithi110 ceph-mon[34830]: pgmap v511: 129 pgs: 129 active+clean; 448 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-11T22:19:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:30 smithi092 ceph-mon[31173]: pgmap v511: 129 pgs: 129 active+clean; 448 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-11T22:19:31.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:30 smithi153 ceph-mon[35045]: pgmap v511: 129 pgs: 129 active+clean; 448 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-11T22:19:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:32 smithi110 ceph-mon[34830]: pgmap v512: 129 pgs: 129 active+clean; 448 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-09-11T22:19:33.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:32 smithi092 ceph-mon[31173]: pgmap v512: 129 pgs: 129 active+clean; 448 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-09-11T22:19:33.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:32 smithi153 ceph-mon[35045]: pgmap v512: 129 pgs: 129 active+clean; 448 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-09-11T22:19:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:34 smithi110 ceph-mon[34830]: pgmap v513: 129 pgs: 129 active+clean; 453 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-09-11T22:19:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:34 smithi092 ceph-mon[31173]: pgmap v513: 129 pgs: 129 active+clean; 453 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-09-11T22:19:35.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:34 smithi153 ceph-mon[35045]: pgmap v513: 129 pgs: 129 active+clean; 453 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-09-11T22:19:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:36 smithi110 ceph-mon[34830]: pgmap v514: 129 pgs: 129 active+clean; 456 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-11T22:19:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:36 smithi092 ceph-mon[31173]: pgmap v514: 129 pgs: 129 active+clean; 456 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-11T22:19:37.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:36 smithi153 ceph-mon[35045]: pgmap v514: 129 pgs: 129 active+clean; 456 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-11T22:19:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:38 smithi110 ceph-mon[34830]: pgmap v515: 129 pgs: 129 active+clean; 459 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-11T22:19:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:38 smithi092 ceph-mon[31173]: pgmap v515: 129 pgs: 129 active+clean; 459 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-11T22:19:39.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:38 smithi153 ceph-mon[35045]: pgmap v515: 129 pgs: 129 active+clean; 459 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-11T22:19:40.614 INFO:tasks.workunit.client.0.smithi092.stdout: 65 13218 67 9564 48 4002 96 2024-09-11T22:19:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:40 smithi110 ceph-mon[34830]: pgmap v516: 129 pgs: 129 active+clean; 462 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T22:19:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:40 smithi092 ceph-mon[31173]: pgmap v516: 129 pgs: 129 active+clean; 462 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T22:19:41.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:40 smithi153 ceph-mon[35045]: pgmap v516: 129 pgs: 129 active+clean; 462 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-09-11T22:19:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:42 smithi110 ceph-mon[34830]: pgmap v517: 129 pgs: 129 active+clean; 464 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-11T22:19:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:42 smithi092 ceph-mon[31173]: pgmap v517: 129 pgs: 129 active+clean; 464 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-11T22:19:43.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:42 smithi153 ceph-mon[35045]: pgmap v517: 129 pgs: 129 active+clean; 464 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-09-11T22:19:45.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:44 smithi110 ceph-mon[34830]: pgmap v518: 129 pgs: 129 active+clean; 470 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-11T22:19:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:44 smithi092 ceph-mon[31173]: pgmap v518: 129 pgs: 129 active+clean; 470 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-11T22:19:45.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:44 smithi153 ceph-mon[35045]: pgmap v518: 129 pgs: 129 active+clean; 470 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-11T22:19:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:46 smithi110 ceph-mon[34830]: pgmap v519: 129 pgs: 129 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-11T22:19:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:46 smithi092 ceph-mon[31173]: pgmap v519: 129 pgs: 129 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-11T22:19:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:46 smithi153 ceph-mon[35045]: pgmap v519: 129 pgs: 129 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-11T22:19:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:48 smithi110 ceph-mon[34830]: pgmap v520: 129 pgs: 129 active+clean; 474 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-09-11T22:19:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:48 smithi092 ceph-mon[31173]: pgmap v520: 129 pgs: 129 active+clean; 474 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-09-11T22:19:49.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:48 smithi153 ceph-mon[35045]: pgmap v520: 129 pgs: 129 active+clean; 474 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-09-11T22:19:50.614 INFO:tasks.workunit.client.0.smithi092.stdout: 66 12797 96 9468 43 4001 107 2024-09-11T22:19:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:50 smithi110 ceph-mon[34830]: pgmap v521: 129 pgs: 129 active+clean; 480 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T22:19:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:50 smithi092 ceph-mon[31173]: pgmap v521: 129 pgs: 129 active+clean; 480 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T22:19:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:50 smithi153 ceph-mon[35045]: pgmap v521: 129 pgs: 129 active+clean; 480 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T22:19:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:52 smithi110 ceph-mon[34830]: pgmap v522: 129 pgs: 129 active+clean; 480 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-11T22:19:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:52 smithi092 ceph-mon[31173]: pgmap v522: 129 pgs: 129 active+clean; 480 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-11T22:19:53.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:52 smithi153 ceph-mon[35045]: pgmap v522: 129 pgs: 129 active+clean; 480 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-11T22:19:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:54 smithi110 ceph-mon[34830]: pgmap v523: 129 pgs: 129 active+clean; 486 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-09-11T22:19:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:54 smithi092 ceph-mon[31173]: pgmap v523: 129 pgs: 129 active+clean; 486 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-09-11T22:19:55.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:54 smithi153 ceph-mon[35045]: pgmap v523: 129 pgs: 129 active+clean; 486 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-09-11T22:19:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:56 smithi110 ceph-mon[34830]: pgmap v524: 129 pgs: 129 active+clean; 489 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-09-11T22:19:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:56 smithi092 ceph-mon[31173]: pgmap v524: 129 pgs: 129 active+clean; 489 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-09-11T22:19:57.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:56 smithi153 ceph-mon[35045]: pgmap v524: 129 pgs: 129 active+clean; 489 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-09-11T22:19:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:19:58 smithi110 ceph-mon[34830]: pgmap v525: 129 pgs: 129 active+clean; 491 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-11T22:19:59.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:19:58 smithi092 ceph-mon[31173]: pgmap v525: 129 pgs: 129 active+clean; 491 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-11T22:19:59.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:19:58 smithi153 ceph-mon[35045]: pgmap v525: 129 pgs: 129 active+clean; 491 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-11T22:20:00.614 INFO:tasks.workunit.client.0.smithi092.stdout: 67 12711 74 9424 62 4159 101 2024-09-11T22:20:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:00 smithi110 ceph-mon[34830]: pgmap v526: 129 pgs: 129 active+clean; 497 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-11T22:20:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:00 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:00 smithi110 ceph-mon[34830]: overall HEALTH_OK 2024-09-11T22:20:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:00 smithi092 ceph-mon[31173]: pgmap v526: 129 pgs: 129 active+clean; 497 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-11T22:20:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:00 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:00 smithi092 ceph-mon[31173]: overall HEALTH_OK 2024-09-11T22:20:01.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:00 smithi153 ceph-mon[35045]: pgmap v526: 129 pgs: 129 active+clean; 497 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-11T22:20:01.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:00 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:01.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:00 smithi153 ceph-mon[35045]: overall HEALTH_OK 2024-09-11T22:20:03.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:02 smithi110 ceph-mon[34830]: pgmap v527: 129 pgs: 129 active+clean; 497 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-11T22:20:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:02 smithi092 ceph-mon[31173]: pgmap v527: 129 pgs: 129 active+clean; 497 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-11T22:20:03.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:02 smithi153 ceph-mon[35045]: pgmap v527: 129 pgs: 129 active+clean; 497 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-11T22:20:04.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:03 smithi110 ceph-mon[34830]: pgmap v528: 129 pgs: 129 active+clean; 500 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-11T22:20:04.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:03 smithi092 ceph-mon[31173]: pgmap v528: 129 pgs: 129 active+clean; 500 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-11T22:20:04.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:03 smithi153 ceph-mon[35045]: pgmap v528: 129 pgs: 129 active+clean; 500 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-11T22:20:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:05 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:06.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:05 smithi110 ceph-mon[34830]: pgmap v529: 129 pgs: 129 active+clean; 505 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T22:20:06.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:05 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:06.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:05 smithi092 ceph-mon[31173]: pgmap v529: 129 pgs: 129 active+clean; 505 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T22:20:06.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:05 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:06.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:05 smithi153 ceph-mon[35045]: pgmap v529: 129 pgs: 129 active+clean; 505 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T22:20:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:08 smithi110 ceph-mon[34830]: pgmap v530: 129 pgs: 129 active+clean; 507 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-09-11T22:20:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:20:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:08 smithi092 ceph-mon[31173]: pgmap v530: 129 pgs: 129 active+clean; 507 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-09-11T22:20:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:20:09.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:08 smithi153 ceph-mon[35045]: pgmap v530: 129 pgs: 129 active+clean; 507 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-09-11T22:20:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:20:10.614 INFO:tasks.workunit.client.0.smithi092.stdout: 67 12740 22 9056 69 3541 130 2024-09-11T22:20:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:10 smithi110 ceph-mon[34830]: pgmap v531: 129 pgs: 129 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-11T22:20:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:10 smithi092 ceph-mon[31173]: pgmap v531: 129 pgs: 129 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-11T22:20:11.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:10 smithi153 ceph-mon[35045]: pgmap v531: 129 pgs: 129 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-11T22:20:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:12 smithi110 ceph-mon[34830]: pgmap v532: 129 pgs: 129 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-11T22:20:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:12 smithi092 ceph-mon[31173]: pgmap v532: 129 pgs: 129 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-11T22:20:13.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:12 smithi153 ceph-mon[35045]: pgmap v532: 129 pgs: 129 active+clean; 512 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-11T22:20:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:14 smithi110 ceph-mon[34830]: pgmap v533: 129 pgs: 129 active+clean; 517 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-11T22:20:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:14 smithi092 ceph-mon[31173]: pgmap v533: 129 pgs: 129 active+clean; 517 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-11T22:20:15.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:14 smithi153 ceph-mon[35045]: pgmap v533: 129 pgs: 129 active+clean; 517 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-11T22:20:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:16 smithi110 ceph-mon[34830]: pgmap v534: 129 pgs: 129 active+clean; 520 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-09-11T22:20:17.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:16 smithi092 ceph-mon[31173]: pgmap v534: 129 pgs: 129 active+clean; 520 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-09-11T22:20:17.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:16 smithi153 ceph-mon[35045]: pgmap v534: 129 pgs: 129 active+clean; 520 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-09-11T22:20:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:18 smithi110 ceph-mon[34830]: pgmap v535: 129 pgs: 129 active+clean; 521 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-09-11T22:20:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:18 smithi092 ceph-mon[31173]: pgmap v535: 129 pgs: 129 active+clean; 521 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-09-11T22:20:19.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:18 smithi153 ceph-mon[35045]: pgmap v535: 129 pgs: 129 active+clean; 521 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-09-11T22:20:20.614 INFO:tasks.workunit.client.0.smithi092.stdout: 68 12402 52 9329 45 4454 102 2024-09-11T22:20:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:20 smithi110 ceph-mon[34830]: pgmap v536: 129 pgs: 129 active+clean; 528 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T22:20:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:20 smithi092 ceph-mon[31173]: pgmap v536: 129 pgs: 129 active+clean; 528 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T22:20:21.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:20 smithi153 ceph-mon[35045]: pgmap v536: 129 pgs: 129 active+clean; 528 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T22:20:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:22 smithi110 ceph-mon[34830]: pgmap v537: 129 pgs: 129 active+clean; 528 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-11T22:20:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:22 smithi092 ceph-mon[31173]: pgmap v537: 129 pgs: 129 active+clean; 528 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-11T22:20:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:22 smithi153 ceph-mon[35045]: pgmap v537: 129 pgs: 129 active+clean; 528 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-11T22:20:25.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:24 smithi110 ceph-mon[34830]: pgmap v538: 129 pgs: 129 active+clean; 532 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-11T22:20:25.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:24 smithi092 ceph-mon[31173]: pgmap v538: 129 pgs: 129 active+clean; 532 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-11T22:20:25.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:24 smithi153 ceph-mon[35045]: pgmap v538: 129 pgs: 129 active+clean; 532 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-11T22:20:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:26 smithi110 ceph-mon[34830]: pgmap v539: 129 pgs: 129 active+clean; 535 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-09-11T22:20:27.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:26 smithi092 ceph-mon[31173]: pgmap v539: 129 pgs: 129 active+clean; 535 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-09-11T22:20:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:26 smithi153 ceph-mon[35045]: pgmap v539: 129 pgs: 129 active+clean; 535 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-09-11T22:20:29.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:28 smithi092 ceph-mon[31173]: pgmap v540: 129 pgs: 129 active+clean; 539 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-09-11T22:20:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:28 smithi110 ceph-mon[34830]: pgmap v540: 129 pgs: 129 active+clean; 539 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-09-11T22:20:29.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:28 smithi153 ceph-mon[35045]: pgmap v540: 129 pgs: 129 active+clean; 539 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-09-11T22:20:30.614 INFO:tasks.workunit.client.0.smithi092.stdout: 69 12285 56 9009 61 3965 103 2024-09-11T22:20:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:30 smithi110 ceph-mon[34830]: pgmap v541: 129 pgs: 129 active+clean; 543 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-09-11T22:20:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:30 smithi092 ceph-mon[31173]: pgmap v541: 129 pgs: 129 active+clean; 543 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-09-11T22:20:31.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:30 smithi153 ceph-mon[35045]: pgmap v541: 129 pgs: 129 active+clean; 543 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-09-11T22:20:32.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:32 smithi092 ceph-mon[31173]: pgmap v542: 129 pgs: 129 active+clean; 543 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-11T22:20:32.538 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:32 smithi153 ceph-mon[35045]: pgmap v542: 129 pgs: 129 active+clean; 543 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-11T22:20:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:32 smithi110 ceph-mon[34830]: pgmap v542: 129 pgs: 129 active+clean; 543 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-11T22:20:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:34 smithi110 ceph-mon[34830]: pgmap v543: 129 pgs: 129 active+clean; 551 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-11T22:20:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:34 smithi092 ceph-mon[31173]: pgmap v543: 129 pgs: 129 active+clean; 551 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-11T22:20:35.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:34 smithi153 ceph-mon[35045]: pgmap v543: 129 pgs: 129 active+clean; 551 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-11T22:20:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:36 smithi110 ceph-mon[34830]: pgmap v544: 129 pgs: 129 active+clean; 554 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-11T22:20:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:36 smithi092 ceph-mon[31173]: pgmap v544: 129 pgs: 129 active+clean; 554 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-11T22:20:37.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:36 smithi153 ceph-mon[35045]: pgmap v544: 129 pgs: 129 active+clean; 554 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-11T22:20:39.137 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:38 smithi092 ceph-mon[31173]: pgmap v545: 129 pgs: 129 active+clean; 555 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-11T22:20:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:38 smithi110 ceph-mon[34830]: pgmap v545: 129 pgs: 129 active+clean; 555 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-11T22:20:39.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:38 smithi153 ceph-mon[35045]: pgmap v545: 129 pgs: 129 active+clean; 555 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-11T22:20:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:40 smithi110 ceph-mon[34830]: pgmap v546: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-11T22:20:41.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:40 smithi092 ceph-mon[31173]: pgmap v546: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-11T22:20:41.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:40 smithi153 ceph-mon[35045]: pgmap v546: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-11T22:20:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:42 smithi110 ceph-mon[34830]: pgmap v547: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-09-11T22:20:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:42 smithi092 ceph-mon[31173]: pgmap v547: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-09-11T22:20:43.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:42 smithi153 ceph-mon[35045]: pgmap v547: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-09-11T22:20:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:44 smithi110 ceph-mon[34830]: pgmap v548: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-09-11T22:20:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:44 smithi092 ceph-mon[31173]: pgmap v548: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-09-11T22:20:45.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:44 smithi153 ceph-mon[35045]: pgmap v548: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-09-11T22:20:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:46 smithi110 ceph-mon[34830]: pgmap v549: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 557 KiB/s wr, 96 op/s 2024-09-11T22:20:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:46 smithi092 ceph-mon[31173]: pgmap v549: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 557 KiB/s wr, 96 op/s 2024-09-11T22:20:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:46 smithi153 ceph-mon[35045]: pgmap v549: 129 pgs: 129 active+clean; 556 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 557 KiB/s wr, 96 op/s 2024-09-11T22:20:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:48 smithi110 ceph-mon[34830]: pgmap v550: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 249 KiB/s wr, 62 op/s 2024-09-11T22:20:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:48 smithi092 ceph-mon[31173]: pgmap v550: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 249 KiB/s wr, 62 op/s 2024-09-11T22:20:49.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:48 smithi153 ceph-mon[35045]: pgmap v550: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 249 KiB/s wr, 62 op/s 2024-09-11T22:20:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:50 smithi110 ceph-mon[34830]: pgmap v551: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-11T22:20:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:50 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:50 smithi092 ceph-mon[31173]: pgmap v551: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-11T22:20:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:50 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:51.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:50 smithi153 ceph-mon[35045]: pgmap v551: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-11T22:20:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:50 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:52 smithi110 ceph-mon[34830]: pgmap v552: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 92 KiB/s wr, 38 op/s 2024-09-11T22:20:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:52 smithi092 ceph-mon[31173]: pgmap v552: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 92 KiB/s wr, 38 op/s 2024-09-11T22:20:53.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:52 smithi153 ceph-mon[35045]: pgmap v552: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 92 KiB/s wr, 38 op/s 2024-09-11T22:20:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:54 smithi110 ceph-mon[34830]: pgmap v553: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 122 KiB/s wr, 46 op/s 2024-09-11T22:20:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:54 smithi092 ceph-mon[31173]: pgmap v553: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 122 KiB/s wr, 46 op/s 2024-09-11T22:20:55.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:54 smithi153 ceph-mon[35045]: pgmap v553: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 122 KiB/s wr, 46 op/s 2024-09-11T22:20:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:56 smithi110 ceph-mon[34830]: pgmap v554: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 133 KiB/s wr, 47 op/s 2024-09-11T22:20:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:56 smithi092 ceph-mon[31173]: pgmap v554: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 133 KiB/s wr, 47 op/s 2024-09-11T22:20:57.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:56 smithi153 ceph-mon[35045]: pgmap v554: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 133 KiB/s wr, 47 op/s 2024-09-11T22:20:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:20:58 smithi110 ceph-mon[34830]: pgmap v555: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 126 KiB/s wr, 41 op/s 2024-09-11T22:20:59.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:20:58 smithi092 ceph-mon[31173]: pgmap v555: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 126 KiB/s wr, 41 op/s 2024-09-11T22:20:59.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:20:58 smithi153 ceph-mon[35045]: pgmap v555: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 126 KiB/s wr, 41 op/s 2024-09-11T22:21:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:00 smithi110 ceph-mon[34830]: pgmap v556: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 124 KiB/s wr, 44 op/s 2024-09-11T22:21:01.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:00 smithi092 ceph-mon[31173]: pgmap v556: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 124 KiB/s wr, 44 op/s 2024-09-11T22:21:01.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:00 smithi153 ceph-mon[35045]: pgmap v556: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 124 KiB/s wr, 44 op/s 2024-09-11T22:21:02.038 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:01 smithi153 ceph-mon[35045]: pgmap v557: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 83 KiB/s wr, 31 op/s 2024-09-11T22:21:02.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:01 smithi092 ceph-mon[31173]: pgmap v557: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 83 KiB/s wr, 31 op/s 2024-09-11T22:21:02.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:01 smithi110 ceph-mon[34830]: pgmap v557: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 83 KiB/s wr, 31 op/s 2024-09-11T22:21:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:04 smithi110 ceph-mon[34830]: pgmap v558: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 88 KiB/s wr, 35 op/s 2024-09-11T22:21:05.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:04 smithi092 ceph-mon[31173]: pgmap v558: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 88 KiB/s wr, 35 op/s 2024-09-11T22:21:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:04 smithi153 ceph-mon[35045]: pgmap v558: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 88 KiB/s wr, 35 op/s 2024-09-11T22:21:07.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:06 smithi110 ceph-mon[34830]: pgmap v559: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 30 op/s 2024-09-11T22:21:07.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:06 smithi092 ceph-mon[31173]: pgmap v559: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 30 op/s 2024-09-11T22:21:07.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:06 smithi153 ceph-mon[35045]: pgmap v559: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 30 op/s 2024-09-11T22:21:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:08 smithi110 ceph-mon[34830]: pgmap v560: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-09-11T22:21:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:08 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:21:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:08 smithi092 ceph-mon[31173]: pgmap v560: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-09-11T22:21:09.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:08 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:21:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:08 smithi153 ceph-mon[35045]: pgmap v560: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-09-11T22:21:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:08 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:21:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:21:10.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:21:10.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:21:10.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:21:10.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:21:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:21:10.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:21:10.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:21:10.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:21:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:10 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T22:21:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:10 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-11T22:21:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:10 smithi110 ceph-mon[34830]: Unable to set osd_memory_target on smithi110 to 184555212: error parsing value: Value '184555212' is below minimum 939524096 2024-09-11T22:21:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:10 smithi110 ceph-mon[34830]: Unable to set osd_memory_target on smithi092 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-11T22:21:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:10 smithi110 ceph-mon[34830]: pgmap v561: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-09-11T22:21:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:10 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T22:21:11.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:10 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-11T22:21:11.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:10 smithi092 ceph-mon[31173]: Unable to set osd_memory_target on smithi110 to 184555212: error parsing value: Value '184555212' is below minimum 939524096 2024-09-11T22:21:11.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:10 smithi092 ceph-mon[31173]: Unable to set osd_memory_target on smithi092 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-11T22:21:11.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:10 smithi092 ceph-mon[31173]: pgmap v561: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-09-11T22:21:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:10 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T22:21:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:10 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-11T22:21:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:10 smithi153 ceph-mon[35045]: Unable to set osd_memory_target on smithi110 to 184555212: error parsing value: Value '184555212' is below minimum 939524096 2024-09-11T22:21:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:10 smithi153 ceph-mon[35045]: Unable to set osd_memory_target on smithi092 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-11T22:21:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:10 smithi153 ceph-mon[35045]: pgmap v561: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-09-11T22:21:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:12 smithi110 ceph-mon[34830]: pgmap v562: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-11T22:21:13.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:12 smithi092 ceph-mon[31173]: pgmap v562: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-11T22:21:13.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:12 smithi153 ceph-mon[35045]: pgmap v562: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-11T22:21:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:14 smithi110 ceph-mon[34830]: pgmap v563: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 KiB/s wr, 25 op/s 2024-09-11T22:21:15.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:14 smithi092 ceph-mon[31173]: pgmap v563: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 KiB/s wr, 25 op/s 2024-09-11T22:21:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:14 smithi153 ceph-mon[35045]: pgmap v563: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 KiB/s wr, 25 op/s 2024-09-11T22:21:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:16 smithi110 ceph-mon[34830]: pgmap v564: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T22:21:17.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:16 smithi092 ceph-mon[31173]: pgmap v564: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T22:21:17.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:16 smithi153 ceph-mon[35045]: pgmap v564: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T22:21:19.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:18 smithi110 ceph-mon[34830]: pgmap v565: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-09-11T22:21:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:18 smithi092 ceph-mon[31173]: pgmap v565: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-09-11T22:21:19.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:18 smithi153 ceph-mon[35045]: pgmap v565: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-09-11T22:21:20.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:19 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:20.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:19 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:20.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:19 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:21.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:20 smithi110 ceph-mon[34830]: pgmap v566: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-09-11T22:21:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:20 smithi092 ceph-mon[31173]: pgmap v566: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-09-11T22:21:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:20 smithi153 ceph-mon[35045]: pgmap v566: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-09-11T22:21:23.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:22 smithi110 ceph-mon[34830]: pgmap v567: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-11T22:21:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:22 smithi092 ceph-mon[31173]: pgmap v567: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-11T22:21:23.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:22 smithi153 ceph-mon[35045]: pgmap v567: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-11T22:21:24.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:23 smithi110 ceph-mon[34830]: pgmap v568: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-09-11T22:21:24.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:23 smithi092 ceph-mon[31173]: pgmap v568: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-09-11T22:21:24.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:23 smithi153 ceph-mon[35045]: pgmap v568: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-09-11T22:21:27.050 INFO:tasks.workunit.client.0.smithi092.stdout: 2024-09-11T22:21:27.050 INFO:tasks.workunit.client.0.smithi092.stdout:Final score for writes: 69 2024-09-11T22:21:27.050 INFO:tasks.workunit.client.0.smithi092.stdout:Final score for reads : 8557 2024-09-11T22:21:27.050 INFO:tasks.workunit.client.0.smithi092.stdout: 2024-09-11T22:21:27.051 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:21:27.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:21:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:26 smithi110 ceph-mon[34830]: pgmap v569: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-11T22:21:27.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:26 smithi092 ceph-mon[31173]: pgmap v569: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-11T22:21:27.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:26 smithi153 ceph-mon[35045]: pgmap v569: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-11T22:21:29.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:28 smithi110 ceph-mon[34830]: pgmap v570: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-11T22:21:29.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:28 smithi092 ceph-mon[31173]: pgmap v570: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-11T22:21:29.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:28 smithi153 ceph-mon[35045]: pgmap v570: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-11T22:21:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:30 smithi110 ceph-mon[34830]: pgmap v571: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 96 KiB/s wr, 27 op/s 2024-09-11T22:21:31.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:30 smithi092 ceph-mon[31173]: pgmap v571: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 96 KiB/s wr, 27 op/s 2024-09-11T22:21:31.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:30 smithi153 ceph-mon[35045]: pgmap v571: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 96 KiB/s wr, 27 op/s 2024-09-11T22:21:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:32 smithi110 ceph-mon[34830]: pgmap v572: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 22 op/s 2024-09-11T22:21:33.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:32 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:33.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:32 smithi092 ceph-mon[31173]: pgmap v572: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 22 op/s 2024-09-11T22:21:33.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:32 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:33.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:32 smithi153 ceph-mon[35045]: pgmap v572: 129 pgs: 129 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 22 op/s 2024-09-11T22:21:33.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:32 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:34.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:33 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:34.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:33 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:34.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:33 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:34 smithi110 ceph-mon[34830]: pgmap v573: 129 pgs: 129 active+clean; 540 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 392 KiB/s wr, 136 op/s 2024-09-11T22:21:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:34 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:34 smithi092 ceph-mon[31173]: pgmap v573: 129 pgs: 129 active+clean; 540 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 392 KiB/s wr, 136 op/s 2024-09-11T22:21:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:34 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:35.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:34 smithi153 ceph-mon[35045]: pgmap v573: 129 pgs: 129 active+clean; 540 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 392 KiB/s wr, 136 op/s 2024-09-11T22:21:35.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:34 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:36 smithi110 ceph-mon[34830]: pgmap v574: 129 pgs: 129 active+clean; 534 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 501 KiB/s wr, 171 op/s 2024-09-11T22:21:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:36 smithi092 ceph-mon[31173]: pgmap v574: 129 pgs: 129 active+clean; 534 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 501 KiB/s wr, 171 op/s 2024-09-11T22:21:37.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:36 smithi153 ceph-mon[35045]: pgmap v574: 129 pgs: 129 active+clean; 534 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 501 KiB/s wr, 171 op/s 2024-09-11T22:21:38.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:37 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:38.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:37 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:38.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:37 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:38 smithi110 ceph-mon[34830]: pgmap v575: 129 pgs: 129 active+clean; 527 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 587 KiB/s wr, 209 op/s 2024-09-11T22:21:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:38 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:38 smithi092 ceph-mon[31173]: pgmap v575: 129 pgs: 129 active+clean; 527 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 587 KiB/s wr, 209 op/s 2024-09-11T22:21:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:38 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:39.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:38 smithi153 ceph-mon[35045]: pgmap v575: 129 pgs: 129 active+clean; 527 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 587 KiB/s wr, 209 op/s 2024-09-11T22:21:39.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:38 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:40.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:39 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:40.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:39 smithi110 ceph-mon[34830]: pgmap v576: 129 pgs: 129 active+clean; 512 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 934 KiB/s wr, 324 op/s 2024-09-11T22:21:40.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:39 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:40.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:39 smithi092 ceph-mon[31173]: pgmap v576: 129 pgs: 129 active+clean; 512 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 934 KiB/s wr, 324 op/s 2024-09-11T22:21:40.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:39 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:40.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:39 smithi153 ceph-mon[35045]: pgmap v576: 129 pgs: 129 active+clean; 512 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 934 KiB/s wr, 324 op/s 2024-09-11T22:21:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:42 smithi110 ceph-mon[34830]: pgmap v577: 129 pgs: 129 active+clean; 512 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 859 KiB/s wr, 312 op/s 2024-09-11T22:21:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:42 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:42 smithi092 ceph-mon[31173]: pgmap v577: 129 pgs: 129 active+clean; 512 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 859 KiB/s wr, 312 op/s 2024-09-11T22:21:43.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:42 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:43.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:42 smithi153 ceph-mon[35045]: pgmap v577: 129 pgs: 129 active+clean; 512 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 859 KiB/s wr, 312 op/s 2024-09-11T22:21:43.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:42 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:44.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:43 smithi110 ceph-mon[34830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:44.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:43 smithi092 ceph-mon[31173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:44.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:43 smithi153 ceph-mon[35045]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:21:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:44 smithi110 ceph-mon[34830]: pgmap v578: 129 pgs: 129 active+clean; 498 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 411 op/s 2024-09-11T22:21:45.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:44 smithi092 ceph-mon[31173]: pgmap v578: 129 pgs: 129 active+clean; 498 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 411 op/s 2024-09-11T22:21:45.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:44 smithi153 ceph-mon[35045]: pgmap v578: 129 pgs: 129 active+clean; 498 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 411 op/s 2024-09-11T22:21:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:46 smithi110 ceph-mon[34830]: pgmap v579: 129 pgs: 129 active+clean; 492 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 895 KiB/s wr, 344 op/s 2024-09-11T22:21:47.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:46 smithi092 ceph-mon[31173]: pgmap v579: 129 pgs: 129 active+clean; 492 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 895 KiB/s wr, 344 op/s 2024-09-11T22:21:47.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:46 smithi153 ceph-mon[35045]: pgmap v579: 129 pgs: 129 active+clean; 492 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 895 KiB/s wr, 344 op/s 2024-09-11T22:21:49.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:48 smithi110 ceph-mon[34830]: pgmap v580: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 780 KiB/s wr, 345 op/s 2024-09-11T22:21:49.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:48 smithi092 ceph-mon[31173]: pgmap v580: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 780 KiB/s wr, 345 op/s 2024-09-11T22:21:49.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:48 smithi153 ceph-mon[35045]: pgmap v580: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 780 KiB/s wr, 345 op/s 2024-09-11T22:21:51.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:50 smithi110 ceph-mon[34830]: pgmap v581: 129 pgs: 129 active+clean; 467 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-11T22:21:51.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:50 smithi092 ceph-mon[31173]: pgmap v581: 129 pgs: 129 active+clean; 467 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-11T22:21:51.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:50 smithi153 ceph-mon[35045]: pgmap v581: 129 pgs: 129 active+clean; 467 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-11T22:21:53.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:52 smithi110 ceph-mon[34830]: pgmap v582: 129 pgs: 129 active+clean; 467 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 746 KiB/s wr, 304 op/s 2024-09-11T22:21:53.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:52 smithi092 ceph-mon[31173]: pgmap v582: 129 pgs: 129 active+clean; 467 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 746 KiB/s wr, 304 op/s 2024-09-11T22:21:53.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:52 smithi153 ceph-mon[35045]: pgmap v582: 129 pgs: 129 active+clean; 467 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 746 KiB/s wr, 304 op/s 2024-09-11T22:21:55.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:54 smithi110 ceph-mon[34830]: pgmap v583: 129 pgs: 129 active+clean; 453 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 900 KiB/s wr, 391 op/s 2024-09-11T22:21:55.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:54 smithi092 ceph-mon[31173]: pgmap v583: 129 pgs: 129 active+clean; 453 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 900 KiB/s wr, 391 op/s 2024-09-11T22:21:55.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:54 smithi153 ceph-mon[35045]: pgmap v583: 129 pgs: 129 active+clean; 453 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 900 KiB/s wr, 391 op/s 2024-09-11T22:21:57.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:56 smithi110 ceph-mon[34830]: pgmap v584: 129 pgs: 129 active+clean; 450 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 967 KiB/s wr, 339 op/s 2024-09-11T22:21:57.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:56 smithi092 ceph-mon[31173]: pgmap v584: 129 pgs: 129 active+clean; 450 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 967 KiB/s wr, 339 op/s 2024-09-11T22:21:57.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:56 smithi153 ceph-mon[35045]: pgmap v584: 129 pgs: 129 active+clean; 450 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 967 KiB/s wr, 339 op/s 2024-09-11T22:21:59.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:21:58 smithi110 ceph-mon[34830]: pgmap v585: 129 pgs: 129 active+clean; 440 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 857 KiB/s wr, 333 op/s 2024-09-11T22:21:59.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:21:58 smithi092 ceph-mon[31173]: pgmap v585: 129 pgs: 129 active+clean; 440 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 857 KiB/s wr, 333 op/s 2024-09-11T22:21:59.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:21:58 smithi153 ceph-mon[35045]: pgmap v585: 129 pgs: 129 active+clean; 440 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 857 KiB/s wr, 333 op/s 2024-09-11T22:22:01.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:00 smithi110 ceph-mon[34830]: pgmap v586: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 414 op/s 2024-09-11T22:22:01.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:00 smithi092 ceph-mon[31173]: pgmap v586: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 414 op/s 2024-09-11T22:22:01.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:00 smithi153 ceph-mon[35045]: pgmap v586: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 414 op/s 2024-09-11T22:22:03.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:02 smithi110 ceph-mon[34830]: pgmap v587: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 794 KiB/s wr, 297 op/s 2024-09-11T22:22:03.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:02 smithi092 ceph-mon[31173]: pgmap v587: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 794 KiB/s wr, 297 op/s 2024-09-11T22:22:03.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:02 smithi153 ceph-mon[35045]: pgmap v587: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 794 KiB/s wr, 297 op/s 2024-09-11T22:22:05.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:04 smithi110 ceph-mon[34830]: pgmap v588: 129 pgs: 129 active+clean; 406 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 951 KiB/s wr, 390 op/s 2024-09-11T22:22:05.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:04 smithi092 ceph-mon[31173]: pgmap v588: 129 pgs: 129 active+clean; 406 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 951 KiB/s wr, 390 op/s 2024-09-11T22:22:05.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:04 smithi153 ceph-mon[35045]: pgmap v588: 129 pgs: 129 active+clean; 406 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 951 KiB/s wr, 390 op/s 2024-09-11T22:22:06.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:05 smithi092 ceph-mon[31173]: pgmap v589: 129 pgs: 129 active+clean; 402 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.0 MiB/s wr, 364 op/s 2024-09-11T22:22:06.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:05 smithi153 ceph-mon[35045]: pgmap v589: 129 pgs: 129 active+clean; 402 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.0 MiB/s wr, 364 op/s 2024-09-11T22:22:06.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:05 smithi110 ceph-mon[34830]: pgmap v589: 129 pgs: 129 active+clean; 402 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.0 MiB/s wr, 364 op/s 2024-09-11T22:22:07.357 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-11T22:22:07.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T22:22:07.734 DEBUG:teuthology.parallel:result is None 2024-09-11T22:22:07.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T22:22:07.766 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T22:22:07.766 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T22:22:07.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T22:22:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 -- 172.21.15.92:0/2954223634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081025e0 msgr2=0x7fa608108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 --2- 172.21.15.92:0/2954223634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081025e0 0x7fa608108e70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa600009ca0 tx=0x7fa60002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 -- 172.21.15.92:0/2954223634 shutdown_connections 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 --2- 172.21.15.92:0/2954223634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6081025e0 0x7fa608108e70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 --2- 172.21.15.92:0/2954223634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa608101c20 0x7fa6081020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 --2- 172.21.15.92:0/2954223634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa608100a30 0x7fa608100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 -- 172.21.15.92:0/2954223634 >> 172.21.15.92:0/2954223634 conn(0x7fa6080fc1b0 msgr2=0x7fa6080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 -- 172.21.15.92:0/2954223634 shutdown_connections 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 -- 172.21.15.92:0/2954223634 wait complete. 2024-09-11T22:22:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 Processor -- start 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 -- start start 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa608100a30 0x7fa608196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.907+0000 7fa61016d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa608101c20 0x7fa608197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa61016d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6081025e0 0x7fa60819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa61016d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810c540 con 0x7fa608100a30 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa61016d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810c3c0 con 0x7fa608101c20 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa61016d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810c6c0 con 0x7fa6081025e0 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa608100a30 0x7fa608196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa608100a30 0x7fa608196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59966/0 (socket says 172.21.15.92:59966) 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 -- 172.21.15.92:0/3827525880 learned_addr learned my addr 172.21.15.92:0/3827525880 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60d708640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa608101c20 0x7fa608197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 -- 172.21.15.92:0/3827525880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6081025e0 msgr2=0x7fa60819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6081025e0 0x7fa60819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 -- 172.21.15.92:0/3827525880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa608101c20 msgr2=0x7fa608197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa608101c20 0x7fa608197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 -- 172.21.15.92:0/3827525880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60819e810 con 0x7fa608100a30 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa60df09640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa608100a30 0x7fa608196bf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d950 tx=0x7fa60400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa5feffd640 1 -- 172.21.15.92:0/3827525880 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604017020 con 0x7fa608100a30 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa5feffd640 1 -- 172.21.15.92:0/3827525880 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa604009c30 con 0x7fa608100a30 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60819eb00 con 0x7fa608100a30 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa5feffd640 1 -- 172.21.15.92:0/3827525880 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60400f6b0 con 0x7fa608100a30 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.908+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60819f0a0 con 0x7fa608100a30 2024-09-11T22:22:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.909+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa608107e40 con 0x7fa608100a30 2024-09-11T22:22:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.910+0000 7fa5feffd640 1 -- 172.21.15.92:0/3827525880 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa60401a020 con 0x7fa608100a30 2024-09-11T22:22:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.911+0000 7fa5feffd640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5f40a2c10 0x7fa5f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.911+0000 7fa60d708640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5f40a2c10 0x7fa5f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.911+0000 7fa5feffd640 1 -- 172.21.15.92:0/3827525880 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa6040cf010 con 0x7fa608100a30 2024-09-11T22:22:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.912+0000 7fa60d708640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5f40a2c10 0x7fa5f40a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa608198100 tx=0x7fa5f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:07.915+0000 7fa5feffd640 1 -- 172.21.15.92:0/3827525880 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa604009de0 con 0x7fa608100a30 2024-09-11T22:22:08.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.084+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa608062ef0 con 0x7fa608100a30 2024-09-11T22:22:08.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.085+0000 7fa5feffd640 1 -- 172.21.15.92:0/3827525880 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24831 (secure 0 0 0) 0x7fa60409bbe0 con 0x7fa608100a30 2024-09-11T22:22:08.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:22:08.086 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:14:17.777929+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5f40a2c10 msgr2=0x7fa5f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5f40a2c10 0x7fa5f40a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa608198100 tx=0x7fa5f8005f90 comp rx=0 tx=0).stop 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa608100a30 msgr2=0x7fa608196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa608100a30 0x7fa608196bf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d950 tx=0x7fa60400de10 comp rx=0 tx=0).stop 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 shutdown_connections 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fa5f40a2c10 0x7fa5f40a50d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6081025e0 0x7fa60819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa608101c20 0x7fa608197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.087+0000 7fa61016d640 1 --2- 172.21.15.92:0/3827525880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa608100a30 0x7fa608196bf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.088+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 >> 172.21.15.92:0/3827525880 conn(0x7fa6080fc1b0 msgr2=0x7fa608105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.088+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 shutdown_connections 2024-09-11T22:22:08.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.088+0000 7fa61016d640 1 -- 172.21.15.92:0/3827525880 wait complete. 2024-09-11T22:22:08.099 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:07:29.899850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T22:11:59.645581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T22:12:01.559996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-11T22:12:08.023536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-11T22:22:08.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T22:22:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- 172.21.15.92:0/2278439074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c101b10 msgr2=0x7f469c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 --2- 172.21.15.92:0/2278439074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c101b10 0x7f469c101f10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f468c009900 tx=0x7f468c02a410 comp rx=0 tx=0).stop 2024-09-11T22:22:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- 172.21.15.92:0/2278439074 shutdown_connections 2024-09-11T22:22:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 --2- 172.21.15.92:0/2278439074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f469c1036c0 0x7f469c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 --2- 172.21.15.92:0/2278439074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c102d00 0x7f469c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 --2- 172.21.15.92:0/2278439074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c101b10 0x7f469c101f10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- 172.21.15.92:0/2278439074 >> 172.21.15.92:0/2278439074 conn(0x7f469c0fd2b0 msgr2=0x7f469c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- 172.21.15.92:0/2278439074 shutdown_connections 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- 172.21.15.92:0/2278439074 wait complete. 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 Processor -- start 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- start start 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f469c101b10 0x7f469c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c102d00 0x7f469c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c1036c0 0x7f469c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10d6a0 con 0x7f469c1036c0 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10d520 con 0x7f469c101b10 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.203+0000 7f46a453b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10d820 con 0x7f469c102d00 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c1036c0 0x7f469c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c1036c0 0x7f469c19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59986/0 (socket says 172.21.15.92:59986) 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 -- 172.21.15.92:0/2218455078 learned_addr learned my addr 172.21.15.92:0/2218455078 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a22d7640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f469c101b10 0x7f469c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a1ad6640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c102d00 0x7f469c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 -- 172.21.15.92:0/2218455078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c102d00 msgr2=0x7f469c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c102d00 0x7f469c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 -- 172.21.15.92:0/2218455078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f469c101b10 msgr2=0x7f469c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f469c101b10 0x7f469c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 -- 172.21.15.92:0/2218455078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c19fa90 con 0x7f469c1036c0 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a2ad8640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c1036c0 0x7f469c19f420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f469400eaa0 tx=0x7f469400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46937fe640 1 -- 172.21.15.92:0/2218455078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400cdb0 con 0x7f469c1036c0 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46937fe640 1 -- 172.21.15.92:0/2218455078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4694005020 con 0x7f469c1036c0 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46937fe640 1 -- 172.21.15.92:0/2218455078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940116d0 con 0x7f469c1036c0 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c19fd80 con 0x7f469c1036c0 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.204+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c1a02a0 con 0x7f469c1036c0 2024-09-11T22:22:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.205+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c109030 con 0x7f469c1036c0 2024-09-11T22:22:08.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.206+0000 7f46937fe640 1 -- 172.21.15.92:0/2218455078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46940051d0 con 0x7f469c1036c0 2024-09-11T22:22:08.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.207+0000 7f46937fe640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f46880a2b90 0x7f46880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.207+0000 7f46a22d7640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f46880a2b90 0x7f46880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.208+0000 7f46937fe640 1 -- 172.21.15.92:0/2218455078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f46940cf970 con 0x7f469c1036c0 2024-09-11T22:22:08.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.208+0000 7f46a22d7640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f46880a2b90 0x7f46880a5050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f468c004770 tx=0x7f468c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.211+0000 7f46937fe640 1 -- 172.21.15.92:0/2218455078 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4694016030 con 0x7f469c1036c0 2024-09-11T22:22:08.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.378+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f469c062e90 con 0x7f469c1036c0 2024-09-11T22:22:08.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.378+0000 7f46937fe640 1 -- 172.21.15.92:0/2218455078 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f469409c4e0 con 0x7f469c1036c0 2024-09-11T22:22:08.379 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-09-11T22:22:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f46880a2b90 msgr2=0x7f46880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f46880a2b90 0x7f46880a5050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f468c004770 tx=0x7f468c000bc0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c1036c0 msgr2=0x7f469c19f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c1036c0 0x7f469c19f420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f469400eaa0 tx=0x7f469400ef60 comp rx=0 tx=0).stop 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 shutdown_connections 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f46880a2b90 0x7f46880a5050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f469c1036c0 0x7f469c19f420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c102d00 0x7f469c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 --2- 172.21.15.92:0/2218455078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f469c101b10 0x7f469c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 >> 172.21.15.92:0/2218455078 conn(0x7f469c0fd2b0 msgr2=0x7f469c106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 shutdown_connections 2024-09-11T22:22:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.381+0000 7f46a453b640 1 -- 172.21.15.92:0/2218455078 wait complete. 2024-09-11T22:22:08.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T22:22:08.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 -- 172.21.15.92:0/166529446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0210102590 msgr2=0x7f0210104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 --2- 172.21.15.92:0/166529446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0210102590 0x7f0210104980 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0200009900 tx=0x7f020002a410 comp rx=0 tx=0).stop 2024-09-11T22:22:08.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 -- 172.21.15.92:0/166529446 shutdown_connections 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 --2- 172.21.15.92:0/166529446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02101077f0 0x7f0210109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 --2- 172.21.15.92:0/166529446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0210104ec0 0x7f02101072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 --2- 172.21.15.92:0/166529446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0210102590 0x7f0210104980 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 -- 172.21.15.92:0/166529446 >> 172.21.15.92:0/166529446 conn(0x7f02100fc1b0 msgr2=0x7f02100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 -- 172.21.15.92:0/166529446 shutdown_connections 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.494+0000 7f02185f6640 1 -- 172.21.15.92:0/166529446 wait complete. 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 Processor -- start 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 -- start start 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0210102590 0x7f0210101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0210104ec0 0x7f0210100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02101077f0 0x7f0210100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f021010d6a0 con 0x7f02101077f0 2024-09-11T22:22:08.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f021010d520 con 0x7f0210104ec0 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f02185f6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f021010d820 con 0x7f0210102590 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02101077f0 0x7f0210100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02101077f0 0x7f0210100a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60000/0 (socket says 172.21.15.92:60000) 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 -- 172.21.15.92:0/4126956286 learned_addr learned my addr 172.21.15.92:0/4126956286 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216392640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0210102590 0x7f0210101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 -- 172.21.15.92:0/4126956286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0210102590 msgr2=0x7f0210101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0210102590 0x7f0210101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 -- 172.21.15.92:0/4126956286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0210104ec0 msgr2=0x7f0210100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0210104ec0 0x7f0210100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.495+0000 7f0216b93640 1 -- 172.21.15.92:0/4126956286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0210101310 con 0x7f02101077f0 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.496+0000 7f0216b93640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02101077f0 0x7f0210100a80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f020800e9d0 tx=0x7f020800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.496+0000 7f02077fe640 1 -- 172.21.15.92:0/4126956286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020800cd80 con 0x7f02101077f0 2024-09-11T22:22:08.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.496+0000 7f02077fe640 1 -- 172.21.15.92:0/4126956286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0208005020 con 0x7f02101077f0 2024-09-11T22:22:08.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.496+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02101a1f60 con 0x7f02101077f0 2024-09-11T22:22:08.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.496+0000 7f02077fe640 1 -- 172.21.15.92:0/4126956286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080116d0 con 0x7f02101077f0 2024-09-11T22:22:08.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.496+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101a23d0 con 0x7f02101077f0 2024-09-11T22:22:08.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.497+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02100fdca0 con 0x7f02101077f0 2024-09-11T22:22:08.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.498+0000 7f02077fe640 1 -- 172.21.15.92:0/4126956286 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02080051d0 con 0x7f02101077f0 2024-09-11T22:22:08.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.498+0000 7f02077fe640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01fc0a2b90 0x7f01fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.499+0000 7f02077fe640 1 -- 172.21.15.92:0/4126956286 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f02080cf9a0 con 0x7f02101077f0 2024-09-11T22:22:08.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.499+0000 7f0216392640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01fc0a2b90 0x7f01fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.499+0000 7f0216392640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01fc0a2b90 0x7f01fc0a5050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0200006010 tx=0x7f0200000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.502+0000 7f02077fe640 1 -- 172.21.15.92:0/4126956286 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0208016030 con 0x7f02101077f0 2024-09-11T22:22:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.667+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f021002da10 con 0x7f02101077f0 2024-09-11T22:22:08.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.667+0000 7f02077fe640 1 -- 172.21.15.92:0/4126956286 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f020809c570 con 0x7f02101077f0 2024-09-11T22:22:08.668 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01fc0a2b90 msgr2=0x7f01fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01fc0a2b90 0x7f01fc0a5050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0200006010 tx=0x7f0200000bc0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02101077f0 msgr2=0x7f0210100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02101077f0 0x7f0210100a80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f020800e9d0 tx=0x7f020800ee90 comp rx=0 tx=0).stop 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 shutdown_connections 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f01fc0a2b90 0x7f01fc0a5050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02101077f0 0x7f0210100a80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0210104ec0 0x7f0210100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 --2- 172.21.15.92:0/4126956286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0210102590 0x7f0210101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 >> 172.21.15.92:0/4126956286 conn(0x7f02100fc1b0 msgr2=0x7f0210102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 shutdown_connections 2024-09-11T22:22:08.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.669+0000 7f02185f6640 1 -- 172.21.15.92:0/4126956286 wait complete. 2024-09-11T22:22:08.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 -- 172.21.15.92:0/4062098752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3894102d00 msgr2=0x7f3894103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 --2- 172.21.15.92:0/4062098752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3894102d00 0x7f3894103180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f388402a410 comp rx=0 tx=0).stop 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 -- 172.21.15.92:0/4062098752 shutdown_connections 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 --2- 172.21.15.92:0/4062098752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38941036c0 0x7f3894109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 --2- 172.21.15.92:0/4062098752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3894102d00 0x7f3894103180 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 --2- 172.21.15.92:0/4062098752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3894101b10 0x7f3894101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 -- 172.21.15.92:0/4062098752 >> 172.21.15.92:0/4062098752 conn(0x7f38940fd2b0 msgr2=0x7f38940ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.784+0000 7f389bbc0640 1 -- 172.21.15.92:0/4062098752 shutdown_connections 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 -- 172.21.15.92:0/4062098752 wait complete. 2024-09-11T22:22:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 Processor -- start 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 -- start start 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3894101b10 0x7f3894197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3894102d00 0x7f3894198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38941036c0 0x7f389419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f389410d620 con 0x7f38941036c0 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f389410d4a0 con 0x7f3894101b10 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389bbc0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f389410d7a0 con 0x7f3894102d00 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389a15d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38941036c0 0x7f389419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389a15d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38941036c0 0x7f389419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60020/0 (socket says 172.21.15.92:60020) 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389a15d640 1 -- 172.21.15.92:0/2107419835 learned_addr learned my addr 172.21.15.92:0/2107419835 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.785+0000 7f389995c640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3894101b10 0x7f3894197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389915b640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3894102d00 0x7f3894198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389a15d640 1 -- 172.21.15.92:0/2107419835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3894102d00 msgr2=0x7f3894198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389a15d640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3894102d00 0x7f3894198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389a15d640 1 -- 172.21.15.92:0/2107419835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3894101b10 msgr2=0x7f3894197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389a15d640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3894101b10 0x7f3894197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389a15d640 1 -- 172.21.15.92:0/2107419835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389419f9e0 con 0x7f38941036c0 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389915b640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3894102d00 0x7f3894198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389a15d640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38941036c0 0x7f389419f370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f388c00eaa0 tx=0x7f388c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f388affd640 1 -- 172.21.15.92:0/2107419835 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c00cdb0 con 0x7f38941036c0 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f388affd640 1 -- 172.21.15.92:0/2107419835 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f388c005020 con 0x7f38941036c0 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389419fcd0 con 0x7f38941036c0 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f388affd640 1 -- 172.21.15.92:0/2107419835 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0116d0 con 0x7f38941036c0 2024-09-11T22:22:08.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.786+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38941a01d0 con 0x7f38941036c0 2024-09-11T22:22:08.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.787+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3894109010 con 0x7f38941036c0 2024-09-11T22:22:08.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.788+0000 7f388affd640 1 -- 172.21.15.92:0/2107419835 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f388c0051d0 con 0x7f38941036c0 2024-09-11T22:22:08.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.789+0000 7f388affd640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f38800a2b90 0x7f38800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.789+0000 7f389995c640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f38800a2b90 0x7f38800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.789+0000 7f388affd640 1 -- 172.21.15.92:0/2107419835 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f388c0cf610 con 0x7f38941036c0 2024-09-11T22:22:08.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.789+0000 7f389995c640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f38800a2b90 0x7f38800a5050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3890005d00 tx=0x7f3890005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.793+0000 7f388affd640 1 -- 172.21.15.92:0/2107419835 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f388c016030 con 0x7f38941036c0 2024-09-11T22:22:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.961+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3894062e90 con 0x7f38941036c0 2024-09-11T22:22:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.961+0000 7f388affd640 1 -- 172.21.15.92:0/2107419835 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f388c09c1e0 con 0x7f38941036c0 2024-09-11T22:22:08.962 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-09-11T22:22:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f38800a2b90 msgr2=0x7f38800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f38800a2b90 0x7f38800a5050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3890005d00 tx=0x7f3890005c50 comp rx=0 tx=0).stop 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38941036c0 msgr2=0x7f389419f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38941036c0 0x7f389419f370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f388c00eaa0 tx=0x7f388c00ef60 comp rx=0 tx=0).stop 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 shutdown_connections 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f38800a2b90 0x7f38800a5050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38941036c0 0x7f389419f370 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3894102d00 0x7f3894198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 --2- 172.21.15.92:0/2107419835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3894101b10 0x7f3894197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 >> 172.21.15.92:0/2107419835 conn(0x7f38940fd2b0 msgr2=0x7f38941061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 shutdown_connections 2024-09-11T22:22:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:08.963+0000 7f389bbc0640 1 -- 172.21.15.92:0/2107419835 wait complete. 2024-09-11T22:22:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:08 smithi092 ceph-mon[31173]: pgmap v590: 129 pgs: 129 active+clean; 392 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 744 KiB/s wr, 357 op/s 2024-09-11T22:22:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:08 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3827525880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:08 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2218455078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:22:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:08 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4126956286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:22:08.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-11T22:22:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 -- 172.21.15.92:0/197037497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80100f50 msgr2=0x7efc80101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 --2- 172.21.15.92:0/197037497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80100f50 0x7efc80101350 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efc70009900 tx=0x7efc7002a480 comp rx=0 tx=0).stop 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 -- 172.21.15.92:0/197037497 shutdown_connections 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 --2- 172.21.15.92:0/197037497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc80102620 0x7efc80108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 --2- 172.21.15.92:0/197037497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc80101c60 0x7efc801020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 --2- 172.21.15.92:0/197037497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80100f50 0x7efc80101350 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 -- 172.21.15.92:0/197037497 >> 172.21.15.92:0/197037497 conn(0x7efc800fb950 msgr2=0x7efc800fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 -- 172.21.15.92:0/197037497 shutdown_connections 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.077+0000 7efc849d8640 1 -- 172.21.15.92:0/197037497 wait complete. 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 Processor -- start 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 -- start start 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc80100f50 0x7efc80197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80101c60 0x7efc801983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc80102620 0x7efc8019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010c580 con 0x7efc80101c60 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010c400 con 0x7efc80100f50 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc849d8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010c700 con 0x7efc80102620 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80101c60 0x7efc801983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80101c60 0x7efc801983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60040/0 (socket says 172.21.15.92:60040) 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 -- 172.21.15.92:0/3446152896 learned_addr learned my addr 172.21.15.92:0/3446152896 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc7ed9d640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc80102620 0x7efc8019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc7e59c640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc80100f50 0x7efc80197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 -- 172.21.15.92:0/3446152896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc80102620 msgr2=0x7efc8019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc80102620 0x7efc8019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 -- 172.21.15.92:0/3446152896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc80100f50 msgr2=0x7efc80197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc80100f50 0x7efc80197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.078+0000 7efc77fff640 1 -- 172.21.15.92:0/3446152896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8019fac0 con 0x7efc80101c60 2024-09-11T22:22:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.079+0000 7efc77fff640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80101c60 0x7efc801983e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c970 tx=0x7efc6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.079+0000 7efc777fe640 1 -- 172.21.15.92:0/3446152896 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68007bb0 con 0x7efc80101c60 2024-09-11T22:22:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.079+0000 7efc777fe640 1 -- 172.21.15.92:0/3446152896 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc68007d50 con 0x7efc80101c60 2024-09-11T22:22:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.079+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8019fdb0 con 0x7efc80101c60 2024-09-11T22:22:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.079+0000 7efc777fe640 1 -- 172.21.15.92:0/3446152896 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6800f450 con 0x7efc80101c60 2024-09-11T22:22:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.079+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801a0220 con 0x7efc80101c60 2024-09-11T22:22:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.080+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc801a04b0 con 0x7efc80101c60 2024-09-11T22:22:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.081+0000 7efc777fe640 1 -- 172.21.15.92:0/3446152896 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc68004f40 con 0x7efc80101c60 2024-09-11T22:22:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.081+0000 7efc777fe640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efc6c0a2b90 0x7efc6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.082+0000 7efc777fe640 1 -- 172.21.15.92:0/3446152896 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efc680cf1f0 con 0x7efc80101c60 2024-09-11T22:22:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.082+0000 7efc7e59c640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efc6c0a2b90 0x7efc6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.082+0000 7efc7e59c640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efc6c0a2b90 0x7efc6c0a5050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efc70000c00 tx=0x7efc70032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:09.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.086+0000 7efc777fe640 1 -- 172.21.15.92:0/3446152896 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc68014030 con 0x7efc80101c60 2024-09-11T22:22:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:08 smithi110 ceph-mon[34830]: pgmap v590: 129 pgs: 129 active+clean; 392 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 744 KiB/s wr, 357 op/s 2024-09-11T22:22:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:08 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3827525880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:08 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2218455078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:22:09.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:08 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4126956286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:22:09.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.266+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efc801a0680 con 0x7efc80101c60 2024-09-11T22:22:09.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.266+0000 7efc777fe640 1 -- 172.21.15.92:0/3446152896 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7efc6809bdc0 con 0x7efc80101c60 2024-09-11T22:22:09.267 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-09-11T22:22:09.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efc6c0a2b90 msgr2=0x7efc6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efc6c0a2b90 0x7efc6c0a5050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efc70000c00 tx=0x7efc70032c80 comp rx=0 tx=0).stop 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80101c60 msgr2=0x7efc801983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80101c60 0x7efc801983e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c970 tx=0x7efc6800ce30 comp rx=0 tx=0).stop 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 shutdown_connections 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7efc6c0a2b90 0x7efc6c0a5050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc80102620 0x7efc8019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc80101c60 0x7efc801983e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 --2- 172.21.15.92:0/3446152896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc80100f50 0x7efc80197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 >> 172.21.15.92:0/3446152896 conn(0x7efc800fb950 msgr2=0x7efc800ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 shutdown_connections 2024-09-11T22:22:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.268+0000 7efc849d8640 1 -- 172.21.15.92:0/3446152896 wait complete. 2024-09-11T22:22:09.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T22:22:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:08 smithi153 ceph-mon[35045]: pgmap v590: 129 pgs: 129 active+clean; 392 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 744 KiB/s wr, 357 op/s 2024-09-11T22:22:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:08 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3827525880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:08 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2218455078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:22:09.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:08 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4126956286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:22:09.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 -- 172.21.15.92:0/925512010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca141025c0 msgr2=0x7fca14108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 --2- 172.21.15.92:0/925512010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca141025c0 0x7fca14108e50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009ca0 tx=0x7fca0c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 -- 172.21.15.92:0/925512010 shutdown_connections 2024-09-11T22:22:09.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 --2- 172.21.15.92:0/925512010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca141025c0 0x7fca14108e50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 --2- 172.21.15.92:0/925512010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca14101c00 0x7fca14102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 --2- 172.21.15.92:0/925512010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca14100a10 0x7fca14100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 -- 172.21.15.92:0/925512010 >> 172.21.15.92:0/925512010 conn(0x7fca140fc190 msgr2=0x7fca140fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 -- 172.21.15.92:0/925512010 shutdown_connections 2024-09-11T22:22:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.382+0000 7fca18c65640 1 -- 172.21.15.92:0/925512010 wait complete. 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.383+0000 7fca18c65640 1 Processor -- start 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.383+0000 7fca18c65640 1 -- start start 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca18c65640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca14100a10 0x7fca14196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca18c65640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca14101c00 0x7fca14197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca18c65640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca141025c0 0x7fca1419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca18c65640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca1410c520 con 0x7fca14100a10 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca18c65640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fca1410c3a0 con 0x7fca141025c0 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca18c65640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca1410c6a0 con 0x7fca14101c00 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca1259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca14100a10 0x7fca14196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca1259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca14100a10 0x7fca14196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60064/0 (socket says 172.21.15.92:60064) 2024-09-11T22:22:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca1259c640 1 -- 172.21.15.92:0/2051640579 learned_addr learned my addr 172.21.15.92:0/2051640579 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca12d9d640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca141025c0 0x7fca1419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca0bfff640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca14101c00 0x7fca14197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca12d9d640 1 -- 172.21.15.92:0/2051640579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca14101c00 msgr2=0x7fca14197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca12d9d640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca14101c00 0x7fca14197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca12d9d640 1 -- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca14100a10 msgr2=0x7fca14196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca12d9d640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca14100a10 0x7fca14196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca12d9d640 1 -- 172.21.15.92:0/2051640579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1419e7f0 con 0x7fca141025c0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca1259c640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca14100a10 0x7fca14196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.384+0000 7fca12d9d640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca141025c0 0x7fca1419e180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca0c031040 tx=0x7fca0c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.385+0000 7fca0b7fe640 1 -- 172.21.15.92:0/2051640579 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c004040 con 0x7fca141025c0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.385+0000 7fca0b7fe640 1 -- 172.21.15.92:0/2051640579 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca0c004690 con 0x7fca141025c0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.385+0000 7fca0b7fe640 1 -- 172.21.15.92:0/2051640579 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c030c30 con 0x7fca141025c0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.385+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1419ea80 con 0x7fca141025c0 2024-09-11T22:22:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.385+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1419efd0 con 0x7fca141025c0 2024-09-11T22:22:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.386+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca14107ea0 con 0x7fca141025c0 2024-09-11T22:22:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.387+0000 7fca0b7fe640 1 -- 172.21.15.92:0/2051640579 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca0c0041e0 con 0x7fca141025c0 2024-09-11T22:22:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.388+0000 7fca0b7fe640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca000a2b90 0x7fca000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.388+0000 7fca0b7fe640 1 -- 172.21.15.92:0/2051640579 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca0c0f09a0 con 0x7fca141025c0 2024-09-11T22:22:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.388+0000 7fca1259c640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca000a2b90 0x7fca000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.388+0000 7fca1259c640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca000a2b90 0x7fca000a5050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fca040058c0 tx=0x7fca04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.392+0000 7fca0b7fe640 1 -- 172.21.15.92:0/2051640579 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca0c0bd510 con 0x7fca141025c0 2024-09-11T22:22:09.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.664+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fca1402d9c0 con 0x7fca141025c0 2024-09-11T22:22:09.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.672+0000 7fca0b7fe640 1 -- 172.21.15.92:0/2051640579 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740371 (secure 0 0 0) 0x7fca0c0bce10 con 0x7fca141025c0 2024-09-11T22:22:09.682 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 18, 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "active_gid": 14150, 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "active_name": "x", 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "active_addrs": { 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6800", 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1619835932 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6801", 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1619835932 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "active_addr": "172.21.15.92:6801/1619835932", 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "active_change": "2024-09-11T22:05:15.603553+0000", 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "available": true, 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "standbys": [ 2024-09-11T22:22:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "gid": 24106, 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "z", 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_features": 4540701547738038271, 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.685 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.686 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.687 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.688 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.691 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-09-11T22:22:09.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.693 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.694 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-09-11T22:22:09.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-09-11T22:22:09.697 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-09-11T22:22:09.698 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.700 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.701 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:09.702 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.703 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.704 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-09-11T22:22:09.707 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.708 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.709 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.710 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.713 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:22:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-09-11T22:22:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-09-11T22:22:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-09-11T22:22:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-09-11T22:22:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-09-11T22:22:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-09-11T22:22:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-09-11T22:22:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:22:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-09-11T22:22:09.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.736 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.737 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.739 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba": { 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.746 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.749 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.751 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.753 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.754 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "grafana_dashboards_path": { 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.756 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.757 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.758 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.759 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.762 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.763 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.768 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.769 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-09-11T22:22:09.771 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-09-11T22:22:09.772 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.778 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.779 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.780 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.782 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.783 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.786 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.789 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.790 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.791 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.794 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.795 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.796 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.801 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.804 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.805 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.806 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.807 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:09.808 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.809 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.810 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.811 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.812 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.813 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.814 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.817 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.819 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.821 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:22:09.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:22:09.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:22:09.826 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:22:09.827 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.829 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.833 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.834 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.835 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.839 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:22:09.840 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.841 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.842 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.843 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.844 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.845 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.850 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.851 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.853 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.854 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-11T22:22:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.856 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.857 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.858 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.859 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.863 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.863 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.863 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.864 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.865 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.867 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.868 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:22:09.870 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:22:09.871 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.873 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.874 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.875 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.877 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.878 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.879 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.880 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.881 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.884 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.885 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.886 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.887 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.888 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.889 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-09-11T22:22:09.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-09-11T22:22:09.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-11T22:22:09.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-09-11T22:22:09.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-09-11T22:22:09.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-09-11T22:22:09.895 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.897 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.898 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.899 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.900 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.901 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.903 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.904 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.905 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.906 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.909 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.910 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.912 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.913 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.914 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.915 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:22:09.916 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.917 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.918 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.920 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.921 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.922 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.923 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-09-11T22:22:09.924 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.925 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.927 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.928 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.929 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.931 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.932 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.933 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.934 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.935 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.936 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.937 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.939 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.940 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.942 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.944 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:22:09.945 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.946 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.948 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.949 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.950 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-09-11T22:22:09.952 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.953 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.957 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.958 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.959 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.960 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.964 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.966 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.967 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.968 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:22:09.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.973 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.975 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-09-11T22:22:09.981 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.981 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:09.981 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:09.981 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-09-11T22:22:09.981 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-09-11T22:22:09.981 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.983 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-09-11T22:22:09.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.986 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.987 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:09.988 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.989 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.990 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.991 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.994 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.995 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-11T22:22:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.002 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.004 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.005 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.007 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-09-11T22:22:10.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-11T22:22:10.009 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.010 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.011 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.014 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.020 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.021 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.022 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.024 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.025 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.027 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.030 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.031 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.034 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.037 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.038 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.039 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.041 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.042 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.043 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.044 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.045 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.046 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.047 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.048 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.049 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.051 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.052 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.054 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.055 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.057 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.058 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.059 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.063 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.065 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.066 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-09-11T22:22:10.067 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.068 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.069 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.070 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.073 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.074 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.075 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.077 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.078 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.079 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.080 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.082 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:22:10.083 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.084 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-09-11T22:22:10.085 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-09-11T22:22:10.086 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.087 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.088 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.089 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.090 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.092 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.093 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.094 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.095 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.097 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.098 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.099 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.100 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.101 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.102 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.103 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.104 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.105 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.107 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-09-11T22:22:10.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-09-11T22:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-09-11T22:22:10.110 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:22:10.111 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.113 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.115 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.116 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.117 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.118 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.119 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.120 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.121 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: "gid": 24113, 2024-09-11T22:22:10.124 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "y", 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_features": 4540701547738038271, 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.125 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.129 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.130 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.131 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-09-11T22:22:10.132 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.133 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.134 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-09-11T22:22:10.135 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.136 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.138 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.139 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.143 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:22:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:22:10.145 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:22:10.146 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:22:10.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.148 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.149 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.150 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.152 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:22:10.153 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-09-11T22:22:10.154 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.158 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-09-11T22:22:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.161 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.162 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:22:10.163 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.164 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-09-11T22:22:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.167 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.168 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:22:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.171 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.172 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.173 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.174 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.175 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.176 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.178 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.179 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.180 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-09-11T22:22:10.181 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-09-11T22:22:10.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-09-11T22:22:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba": { 2024-09-11T22:22:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-09-11T22:22:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-09-11T22:22:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-09-11T22:22:10.187 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.191 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "grafana_dashboards_path": { 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.194 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.195 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.196 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.197 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.198 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.199 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.205 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-09-11T22:22:10.209 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.210 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.211 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.212 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.213 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.214 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.215 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.216 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.218 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.222 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.223 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-09-11T22:22:10.224 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.226 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.227 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.228 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.229 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.230 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:22:10.232 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:22:10.233 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:22:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:22:10.235 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:22:10.236 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:22:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.243 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.245 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.246 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.247 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.249 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.254 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.258 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.259 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.260 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.263 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.265 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.266 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.267 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.269 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.270 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.271 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.272 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.273 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.274 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.276 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.277 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.278 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.280 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.281 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.282 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.284 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.285 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.286 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.287 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-11T22:22:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.290 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.291 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.292 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.293 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.295 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-09-11T22:22:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.300 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.301 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-09-11T22:22:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-09-11T22:22:10.303 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-09-11T22:22:10.304 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-11T22:22:10.305 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-09-11T22:22:10.308 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.310 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.311 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.312 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.314 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.315 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.316 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.317 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.319 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.320 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.323 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.324 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.325 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-09-11T22:22:10.327 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-09-11T22:22:10.328 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-11T22:22:10.329 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-09-11T22:22:10.330 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-09-11T22:22:10.331 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-09-11T22:22:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.334 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.336 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.337 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.338 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.341 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.342 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-11T22:22:10.345 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.347 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.348 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.349 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.351 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-09-11T22:22:10.352 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:22:10.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.354 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.357 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:22:10.359 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:22:10.360 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-09-11T22:22:10.361 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.362 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.363 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.365 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.366 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.367 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.368 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.371 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.372 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.373 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.374 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.379 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.380 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.381 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.384 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.385 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.386 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.389 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.392 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.393 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.394 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.395 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.396 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.397 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.398 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.399 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.402 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.403 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.404 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.408 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.409 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.410 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.411 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.413 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.415 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.416 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.418 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.419 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.423 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.425 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.426 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:22:10.428 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:22:10.429 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.430 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.431 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.432 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-11T22:22:10.433 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.434 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.436 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.437 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.438 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.439 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.440 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.441 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-09-11T22:22:10.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-11T22:22:10.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.445 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.446 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.447 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.448 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:10.449 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.450 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.452 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.453 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.454 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.459 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.460 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.461 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.462 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:22:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.464 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.465 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.466 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.468 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.469 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.472 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.473 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.474 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.475 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-09-11T22:22:10.476 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-09-11T22:22:10.477 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-09-11T22:22:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.479 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.482 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.483 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.485 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.486 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.487 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.488 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.489 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.492 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.493 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.497 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.498 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.501 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.502 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.503 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.504 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.505 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.506 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-09-11T22:22:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:22:10.509 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:22:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:22:10.511 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:22:10.512 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.513 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.514 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.515 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.516 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.517 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.518 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.520 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.523 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.525 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.526 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.527 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.528 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.529 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.532 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.534 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.536 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.539 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.542 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.543 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-09-11T22:22:10.545 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.546 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.547 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.548 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.549 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.552 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.553 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.554 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.555 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.556 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "modules": [ 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "dashboard", 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "iostat", 2024-09-11T22:22:10.557 INFO:teuthology.orchestra.run.smithi092.stdout: "nfs", 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "restful" 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.559 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.560 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.562 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-09-11T22:22:10.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-09-11T22:22:10.565 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-09-11T22:22:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.567 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.569 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.571 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.573 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.574 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-11T22:22:10.575 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-09-11T22:22:10.578 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-09-11T22:22:10.579 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.580 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.583 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.584 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.586 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.587 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.592 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.593 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.594 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-09-11T22:22:10.595 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.596 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.597 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.599 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-09-11T22:22:10.600 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-09-11T22:22:10.601 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-09-11T22:22:10.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-09-11T22:22:10.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-09-11T22:22:10.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-09-11T22:22:10.605 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-09-11T22:22:10.606 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.608 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.609 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.610 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.611 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.612 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-09-11T22:22:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-11T22:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-09-11T22:22:10.617 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba": { 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:22:10.618 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.621 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.622 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.624 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.625 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.626 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "grafana_dashboards_path": { 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:22:10.627 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.628 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.629 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.630 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.635 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.636 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.639 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-09-11T22:22:10.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-11T22:22:10.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.647 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.648 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.649 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.650 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.651 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.652 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:22:10.653 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:22:10.654 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:22:10.655 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:22:10.656 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.657 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.659 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.661 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.662 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-09-11T22:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-09-11T22:22:10.664 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.666 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.667 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.668 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.669 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.670 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.671 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.672 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.673 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.674 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.675 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.677 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.678 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.679 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.680 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.681 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.683 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.685 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.686 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.687 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.692 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.693 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.694 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.695 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:22:10.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:22:10.697 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:22:10.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:22:10.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:22:10.701 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:22:10.702 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:22:10.703 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.705 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.707 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.708 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.709 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.712 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.714 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.717 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.718 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.719 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.728 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.729 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.731 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.732 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.733 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.734 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.735 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.736 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-09-11T22:22:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-09-11T22:22:10.738 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-09-11T22:22:10.739 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.740 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.741 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.742 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-09-11T22:22:10.743 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.745 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.749 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-11T22:22:10.750 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-09-11T22:22:10.751 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-11T22:22:10.752 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:22:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.754 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.755 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.759 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.761 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.762 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-09-11T22:22:10.763 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-11T22:22:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-09-11T22:22:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-09-11T22:22:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.769 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.772 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.773 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.776 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.780 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.781 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.782 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.783 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.785 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.786 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.787 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.788 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.789 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.790 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.791 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-09-11T22:22:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-09-11T22:22:10.794 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-09-11T22:22:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-09-11T22:22:10.797 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.799 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.803 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.805 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.806 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.807 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.808 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.810 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.811 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.812 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.813 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.814 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-09-11T22:22:10.815 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.816 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.817 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.818 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.820 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.821 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-09-11T22:22:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.824 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.825 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.829 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.830 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.832 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.833 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.835 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.836 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.838 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.839 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.842 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.843 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.844 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.845 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.846 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.847 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.848 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.849 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-09-11T22:22:10.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-09-11T22:22:10.852 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.854 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.856 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.857 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-09-11T22:22:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:22:10.859 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-09-11T22:22:10.860 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:10.861 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:10.862 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-09-11T22:22:10.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-11T22:22:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-11T22:22:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.882 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-09-11T22:22:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-09-11T22:22:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-09-11T22:22:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-09-11T22:22:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-09-11T22:22:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-09-11T22:22:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-09-11T22:22:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-09-11T22:22:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-09-11T22:22:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-11T22:22:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-11T22:22:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-11T22:22:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:22:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "services": { 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dashboard": "https://172.21.15.92:8443/" 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "always_on_modules": { 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "octopus": [ 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "pacific": [ 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-11T22:22:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "quincy": [ 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-11T22:22:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "reef": [ 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-11T22:22:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "squid": [ 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-11T22:22:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "last_failure_osd_epoch": 3, 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "active_clients": [ 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4100905256 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "libcephsqlite", 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2742597002 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-11T22:22:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3586351080 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1517821039 2024-09-11T22:22:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T22:22:10.997 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:22:10.998 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2107419835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:22:10.998 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3446152896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-11T22:22:10.998 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:10.998 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2051640579' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:22:10.998 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:10.998 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:10.998 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:09 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2107419835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3446152896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2051640579' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:10.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:11.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:11.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:11.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:09 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:22:11.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca000a2b90 msgr2=0x7fca000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca000a2b90 0x7fca000a5050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fca040058c0 tx=0x7fca04005f90 comp rx=0 tx=0).stop 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca141025c0 msgr2=0x7fca1419e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca141025c0 0x7fca1419e180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca0c031040 tx=0x7fca0c003ba0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 shutdown_connections 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fca000a2b90 0x7fca000a5050 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca141025c0 0x7fca1419e180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca14101c00 0x7fca14197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 --2- 172.21.15.92:0/2051640579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca14100a10 0x7fca14196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 >> 172.21.15.92:0/2051640579 conn(0x7fca140fc190 msgr2=0x7fca14105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 shutdown_connections 2024-09-11T22:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:09.677+0000 7fca18c65640 1 -- 172.21.15.92:0/2051640579 wait complete. 2024-09-11T22:22:11.058 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T22:22:11.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T22:22:11.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2107419835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:22:11.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3446152896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-11T22:22:11.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:11.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2051640579' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:22:11.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:11.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:09 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' 2024-09-11T22:22:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:10 smithi110 ceph-mon[34830]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-11T22:22:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:10 smithi110 ceph-mon[34830]: Unable to set osd_memory_target on smithi153 to 184579584: error parsing value: Value '184579584' is below minimum 939524096 2024-09-11T22:22:11.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:10 smithi110 ceph-mon[34830]: pgmap v591: 129 pgs: 129 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 425 op/s 2024-09-11T22:22:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:10 smithi092 ceph-mon[31173]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-11T22:22:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:10 smithi092 ceph-mon[31173]: Unable to set osd_memory_target on smithi153 to 184579584: error parsing value: Value '184579584' is below minimum 939524096 2024-09-11T22:22:11.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:10 smithi092 ceph-mon[31173]: pgmap v591: 129 pgs: 129 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 425 op/s 2024-09-11T22:22:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:10 smithi153 ceph-mon[35045]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-11T22:22:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:10 smithi153 ceph-mon[35045]: Unable to set osd_memory_target on smithi153 to 184579584: error parsing value: Value '184579584' is below minimum 939524096 2024-09-11T22:22:11.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:10 smithi153 ceph-mon[35045]: pgmap v591: 129 pgs: 129 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 425 op/s 2024-09-11T22:22:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:11.789 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:22:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1114, 2024-09-11T22:22:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1114, 2024-09-11T22:22:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 17110573, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 353429, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 7, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.107680150, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.029472215, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.039906133, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.029770157, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1113, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.035720792, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000032094 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 877, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 854, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 36534299, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 319820, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-09-11T22:22:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.098530641, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.024433143, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.056319830, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.017853889, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 853, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.026821536, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000031443 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 618592, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 104784 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 156, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 283, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 8427768, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 345370, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.026033528, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.010472501, 2024-09-11T22:22:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.009690462, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003943725, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 281, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.009230497, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000032848 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 296080 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3311, 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.071096926, 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000021472 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 2909, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 2909, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-09-11T22:22:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 2909, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 1, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 1, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-11T22:22:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-09-11T22:22:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-09-11T22:22:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 60078, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 70, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 6, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-11T22:22:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 65105527, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 65104389, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 60078 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 13, 2024-09-11T22:22:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3205, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 3192, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 11, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 178, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 167, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 2909, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 7689, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 4780, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 53796, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215328 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-11T22:22:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-11T22:22:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 1, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 0, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-09-11T22:22:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1667, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 215191, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 66, 2024-09-11T22:22:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 166, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-11T22:22:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 2091528, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 11764, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 1941, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 1941, 2024-09-11T22:22:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1941, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.185987042, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000611018 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1941, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1942 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 1941, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 1941, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-09-11T22:22:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1037, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 902, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-11T22:22:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-11T22:22:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 94, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 4, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 8, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 8, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-09-11T22:22:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2143, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 61866616, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2143, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2143, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 61866616, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1941, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 60911298, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1941, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 60911298, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1941, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 1941, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1941, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1941, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-11T22:22:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55a5868d0b40": { 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:11.836 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:22:13.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:12 smithi092 ceph-mon[31173]: pgmap v592: 129 pgs: 129 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 694 KiB/s wr, 304 op/s 2024-09-11T22:22:13.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:12 smithi110 ceph-mon[34830]: pgmap v592: 129 pgs: 129 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 694 KiB/s wr, 304 op/s 2024-09-11T22:22:13.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:12 smithi153 ceph-mon[35045]: pgmap v592: 129 pgs: 129 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 694 KiB/s wr, 304 op/s 2024-09-11T22:22:13.336 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-11T22:22:13.336 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-11T22:22:13.336 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-11T22:22:13.336 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T22:22:13.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T22:22:13.584 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:14.122 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:22:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 177, 2024-09-11T22:22:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 174, 2024-09-11T22:22:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 466080, 2024-09-11T22:22:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 106448, 2024-09-11T22:22:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.024055924, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.010857618, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.015800674, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 173, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006891952, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000039837 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 466080, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 106448 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 132, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 160, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 101664, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 3, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.009545492, 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.007298453, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001061288, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 130, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.004930538, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000037927 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 101664 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 4, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 4, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2944, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 448, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.002285743, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000046377, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.002249784, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003984008, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001328002 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-11T22:22:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 148967, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 29, 2024-09-11T22:22:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 928, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 178, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 441089, 2024-09-11T22:22:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 178, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 178, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 441089, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:14.133 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:22:14.788 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T22:22:14.788 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T22:22:14.788 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-11T22:22:14.788 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T22:22:14.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T22:22:15.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:14 smithi092 ceph-mon[31173]: pgmap v593: 129 pgs: 129 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 698 KiB/s wr, 316 op/s 2024-09-11T22:22:15.035 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:15.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:14 smithi110 ceph-mon[34830]: pgmap v593: 129 pgs: 129 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 698 KiB/s wr, 316 op/s 2024-09-11T22:22:15.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:14 smithi153 ceph-mon[35045]: pgmap v593: 129 pgs: 129 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 698 KiB/s wr, 316 op/s 2024-09-11T22:22:15.565 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:22:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1037, 2024-09-11T22:22:15.565 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1012, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 20507387, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 370452, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 7, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.110899788, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.038079183, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.048746713, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.021957959, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1012, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.036657339, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000036222 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 622080, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 106512 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 755, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 883, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 21164420, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 536085, 2024-09-11T22:22:15.567 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.071169574, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.019774578, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.030745694, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.016122929, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 881, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.023275547, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000026419 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.568 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 296256 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 380, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 382, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 21068445, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 120852, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.043338144, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.007238363, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.024867411, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.008772422, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 380, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.012690554, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000033396 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3393, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.068247647, 2024-09-11T22:22:15.571 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000020114 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-11T22:22:15.572 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-11T22:22:15.573 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 3254, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 3254, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 3254, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 1, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-09-11T22:22:15.574 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-09-11T22:22:15.575 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:15.576 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-09-11T22:22:15.577 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:15.578 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 58676, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-11T22:22:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 68, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 6, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 65763999, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 65763400, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.580 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 58676 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 13, 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3530, 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 3517, 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 11, 2024-09-11T22:22:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 196, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 185, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 3254, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 8765, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 5511, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 56536, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215328 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-11T22:22:15.582 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.584 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.585 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-11T22:22:15.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.587 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.591 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.592 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.593 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1669, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.594 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:15.595 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:15.596 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 230567, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 72, 2024-09-11T22:22:15.597 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 166, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-11T22:22:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 2334408, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 13144, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 1964, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-09-11T22:22:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 1964, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1964, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.106987890, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000563639 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1964, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1965 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 1964, 2024-09-11T22:22:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 1964, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1048, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 914, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-09-11T22:22:15.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-11T22:22:15.604 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-11T22:22:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 94, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 4, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 10, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 10, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.606 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-09-11T22:22:15.607 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2168, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 62531874, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2168, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2168, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 62531874, 2024-09-11T22:22:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:15.609 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1964, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 61571740, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1964, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 61571740, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-11T22:22:15.610 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1964, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 1964, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1964, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1964, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:15.612 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x56256cda0b40": { 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:15.613 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:15.614 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:22:16.299 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-11T22:22:16.299 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-11T22:22:16.299 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-11T22:22:16.299 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T22:22:16.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T22:22:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:16.968 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:16 smithi092 ceph-mon[31173]: pgmap v594: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 547 KiB/s wr, 228 op/s 2024-09-11T22:22:17.068 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T22:22:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 4, 2024-09-11T22:22:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 4, 2024-09-11T22:22:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2944, 2024-09-11T22:22:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 448, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.002945668, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000029530, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.002147128, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003930936, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001310312 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 179, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 176, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 466624, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 107728, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.022991210, 2024-09-11T22:22:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.010595808, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.016028585, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 175, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.007401028, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000042291 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 466624, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 107728 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 133, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 160, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 101840, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 3, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.009865135, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.007197142, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001519721, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:22:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 131, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.005375968, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000041037 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 101840 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-11T22:22:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 148967, 2024-09-11T22:22:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 29, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-11T22:22:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 928, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 180, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 441349, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 180, 2024-09-11T22:22:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 180, 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 441349, 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T22:22:17.079 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T22:22:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:16 smithi110 ceph-mon[34830]: pgmap v594: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 547 KiB/s wr, 228 op/s 2024-09-11T22:22:17.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:16 smithi153 ceph-mon[35045]: pgmap v594: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 547 KiB/s wr, 228 op/s 2024-09-11T22:22:17.716 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T22:22:17.716 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T22:22:17.716 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-11T22:22:17.716 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T22:22:17.717 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T22:22:17.979 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:22:18.539 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 287532, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 122571, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 102028568, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 71658474, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 10, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 9.978906185, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 3.842185135, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 8.551685250, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.132685073, 2024-09-11T22:22:18.540 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 122573, 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2.105080327, 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000017174 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 395211, 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.045117679, 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000114 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 717216, 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 142880 2024-09-11T22:22:18.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 6950, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 7173, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2530027, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 32811221, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 13, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 8, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.494954027, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.186574547, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.157273799, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.132649296, 2024-09-11T22:22:18.542 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7175, 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.316238010, 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000044074 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3225, 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000522264, 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000161 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-11T22:22:18.543 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 309808 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 9637, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 9319, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 3069066, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 24466178, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 14, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 8, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.626072558, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.265336371, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.289348257, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.097837837, 2024-09-11T22:22:18.544 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 9311, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.298683527, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000032078 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 11328, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001799710, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000158 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.545 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 9664, 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 6.198998708, 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000641452 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 6784, 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.534671574, 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000078813 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 73123, 2024-09-11T22:22:18.547 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 62946, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 62946, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 174.460866383, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002771595 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 10177, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 129, 2024-09-11T22:22:18.548 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 13, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 30, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 3, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 4, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 23, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 25, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 2583, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 2, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 2, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 4, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 146530, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 64809, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 10175, 2024-09-11T22:22:18.549 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 205, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 13, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 4832, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 149, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 1444, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 58, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 3570, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 4, 2024-09-11T22:22:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 234, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 4, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 2582, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 6483, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 6, 2024-09-11T22:22:18.551 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 185790, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 1993, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 201, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 12063 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 214, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 160, 2024-09-11T22:22:18.552 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 1, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 3157, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 3156, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:18.553 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 151, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 13, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 67732, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 67732, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T22:22:18.554 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 80, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 80, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 7, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 74987671, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4558, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 11.997605907, 2024-09-11T22:22:18.555 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002632208 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 34, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 3992, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 3958, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 34, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 273, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 239, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 72, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 9905, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 9833, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 2, 2024-09-11T22:22:18.556 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 3980, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 3978, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 81860, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 231712 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 85349, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 1235, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 73123, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 122, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 6820, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1813, 2024-09-11T22:22:18.557 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 59.914448706, 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.033047131 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 422, 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.130307520, 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000308785 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 55191, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 18.947625386, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000343310 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.560 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.561 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 18, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.022093936, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001227440 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 941, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.338414127, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000359632 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 268, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.462173693, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001724528 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1440, 2024-09-11T22:22:18.563 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 89.932179174, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.062452902 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.564 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 10, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.076476706, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.007647670 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001442353, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000721176 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.566 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 8, 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.021709152, 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002713644 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T22:22:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.002405692, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000801897 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2830, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 4.611589938, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001629537 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.568 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 3, 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 2, 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-11T22:22:18.569 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 9205, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 9205, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 479, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:18.570 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:18.571 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:18.572 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 311440, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 122, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 166, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:18.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 181987, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 1567, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T22:22:18.574 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 12736, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 73109174, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 37, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 12736, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 12736, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 48.259097528, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003789187 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 12736, 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 13546 2024-09-11T22:22:18.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 12736, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 298, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 12438, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 11, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 298, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 4856, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 218, 2024-09-11T22:22:18.576 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 13, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 6828, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:18.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 22, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 6, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T22:22:18.578 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 94, 2024-09-11T22:22:18.579 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 10, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 955, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 339 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T22:22:18.580 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 6, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 6300, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 3156, 2024-09-11T22:22:18.581 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 304115, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 80235188, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 304115, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 304115, 2024-09-11T22:22:18.582 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 80235188, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:18.583 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 12736, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 73453082, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 12736, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 73453082, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 12736, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 12736, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 12736, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 12736, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.585 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 3156, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 343908, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 3156, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:18.586 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 298, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 343908, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55a63626e960": { 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 67732, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 70793367, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:18.587 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 67732, 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4558, 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 70793367, 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:18.588 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T22:22:18.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:18 smithi110 ceph-mon[34830]: pgmap v595: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 310 KiB/s wr, 167 op/s 2024-09-11T22:22:19.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:18 smithi092 ceph-mon[31173]: pgmap v595: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 310 KiB/s wr, 167 op/s 2024-09-11T22:22:19.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:18 smithi153 ceph-mon[35045]: pgmap v595: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 310 KiB/s wr, 167 op/s 2024-09-11T22:22:19.582 INFO:tasks.check_counter:Daemon mds.b mds.exported=149 2024-09-11T22:22:19.582 INFO:tasks.check_counter:Daemon mds.b mds.imported=58 2024-09-11T22:22:19.582 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=13 2024-09-11T22:22:19.582 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T22:22:19.582 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T22:22:19.832 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1025, 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1026, 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 25509958, 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 325642, 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 7, 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-09-11T22:22:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.090657713, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.028878717, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.031948783, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.022293223, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1024, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.032239921, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000031484 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:22:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 634, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 609, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 25386891, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 243494, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.063401860, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.016741975, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.042634464, 2024-09-11T22:22:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.008057889, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 609, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.023997283, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000039404 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 623328, 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 109232 2024-09-11T22:22:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 671, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 803, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 21140331, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 511637, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.061790803, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.017298881, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.024674773, 2024-09-11T22:22:20.396 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.015174708, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 801, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.023276859, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000029059 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 298576 2024-09-11T22:22:20.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3606, 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.067932104, 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000018838 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:20.398 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 3236, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 3236, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 3236, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-09-11T22:22:20.400 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 2, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-11T22:22:20.401 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 4, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 234, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:20.402 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:22:20.403 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-09-11T22:22:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T22:22:20.405 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 67732, 2024-09-11T22:22:20.406 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 80, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 7, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 74987671, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 74983370, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 67732 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 24, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 3936, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 3912, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 19, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 238, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 219, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 3236, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 9176, 2024-09-11T22:22:20.408 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 5940, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 54576, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 215328 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-09-11T22:22:20.409 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T22:22:20.411 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.412 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.413 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.414 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.415 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.416 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.417 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:20.418 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:20.419 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.420 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.421 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 3, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 2, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1676, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.422 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:20.423 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:20.424 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 222505, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 69, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:20.425 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 166, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 2356300, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 13346, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:20.426 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 2116, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 26, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 4, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 2116, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2116, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.624969641, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000767944 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.427 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2116, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2117 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 2116, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 2116, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 1155, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 958, 2024-09-11T22:22:20.428 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 2, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:20.429 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T22:22:20.430 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 94, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 9, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 9, 2024-09-11T22:22:20.431 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-11T22:22:20.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 2326, 2024-09-11T22:22:20.433 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 71814328, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 2326, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2326, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 71814328, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:20.434 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2116, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 70827998, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2116, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 70827998, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:20.435 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2116, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2116, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2116, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2116, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:20.437 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55dac49c6b40": { 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:20.438 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:20.439 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T22:22:20.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:20 smithi110 ceph-mon[34830]: pgmap v596: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 315 KiB/s wr, 131 op/s 2024-09-11T22:22:21.144 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T22:22:21.145 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-11T22:22:21.145 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-11T22:22:21.145 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T22:22:21.145 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T22:22:21.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:20 smithi092 ceph-mon[31173]: pgmap v596: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 315 KiB/s wr, 131 op/s 2024-09-11T22:22:21.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:20 smithi153 ceph-mon[35045]: pgmap v596: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 315 KiB/s wr, 131 op/s 2024-09-11T22:22:21.391 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:22:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T22:22:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 214387, 2024-09-11T22:22:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 73487, 2024-09-11T22:22:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 72818451, 2024-09-11T22:22:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 52775643, 2024-09-11T22:22:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 17, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 7.426873610, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 2.831387256, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 6.416738829, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.144529513, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 73484, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.316176178, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000017911 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 272327, 2024-09-11T22:22:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.030860815, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000113 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 33549, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 34467, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 15915325, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 42376514, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 8, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2024-09-11T22:22:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 1.715713718, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.690973556, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 1.125302320, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.105201382, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 34468, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.719553366, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000020875 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 59755, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.008382081, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000140 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 638848, 2024-09-11T22:22:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 113312 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 6962, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 7561, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2167790, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 15679076, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 9, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.438767604, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.180239969, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.204093366, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.066188990, 2024-09-11T22:22:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7559, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.218761172, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000028940 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 9139, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001436151, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000157 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 298224 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T22:22:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 8714, 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 4.869919435, 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000558861 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 6670, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.508358352, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000076215 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 51919, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 45943, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 45943, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 170.242204226, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003705509 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 5976, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 83, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 8, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 18, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 2, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 1, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 15, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 15, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 1317, 2024-09-11T22:22:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 1, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 1, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 4, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 109522, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 55750, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 5975, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 333, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 6, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 8059, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 107, 2024-09-11T22:22:21.904 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 647, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 51, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 2870, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 4, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 234, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:21.905 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 17, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 2065, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 5563, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 156653, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 1287, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 224, 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 10431 2024-09-11T22:22:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 360, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 307, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 2899, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 2899, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 110, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T22:22:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 8, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 60078, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 60078, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 70, 2024-09-11T22:22:21.909 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 70, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 6, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 65105527, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4195, 2024-09-11T22:22:21.910 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 10.005721710, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002385154 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 29, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 3247, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 3218, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 43, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 224, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 181, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 264, 2024-09-11T22:22:21.911 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 8305, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 8041, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 1, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 3220, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 3219, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 79852, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 223520 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 64141, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 2412, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 51919, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 159, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 7858, 2024-09-11T22:22:21.912 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1283, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 53.822549910, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.041950545 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001652434, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000550811 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 39997, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 12.450463760, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000311284 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.915 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.916 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:21.916 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.916 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.916 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.917 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 668, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.194388838, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000291001 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T22:22:21.918 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 182, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.136584406, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000750463 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1308, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 99.411597915, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.076002750 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:21.919 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.920 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.922 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2502, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 4.224966963, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001688635 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.923 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 1, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 0, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 8373, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 8373, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 478, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:21.924 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:21.925 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:21.926 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:21.926 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:21.926 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:21.926 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 335378, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 126, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 166, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:21.928 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 337189, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 3766, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 11387, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 62551856, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 33, 2024-09-11T22:22:21.929 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 11387, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 11387, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 34.946348443, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003068968 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 11387, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 12102 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 11387, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 337, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 11050, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T22:22:21.930 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 10, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 337, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 4532, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 246, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 8, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 5824, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 20, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:21.931 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T22:22:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 94, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2024-09-11T22:22:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 816, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 309 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 5, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 5798, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 2899, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 254894, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 67946134, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 254894, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 254894, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 67946134, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.936 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 11387, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 62867847, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 11387, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 62867847, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 11387, 2024-09-11T22:22:21.938 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 11387, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 11387, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 11387, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 2899, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 315991, 2024-09-11T22:22:21.939 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 2899, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 337, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 315991, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55d3eacf4960": { 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:21.940 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 60078, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 60911223, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 60078, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4195, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 60911223, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:21.941 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.942 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T22:22:22.671 INFO:tasks.check_counter:Daemon mds.h mds.exported=107 2024-09-11T22:22:22.671 INFO:tasks.check_counter:Daemon mds.h mds.imported=51 2024-09-11T22:22:22.671 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=8 2024-09-11T22:22:22.671 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T22:22:22.671 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T22:22:22.913 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:22:23.169 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:22 smithi110 ceph-mon[34830]: pgmap v597: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-09-11T22:22:23.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:22 smithi092 ceph-mon[31173]: pgmap v597: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-09-11T22:22:23.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:22 smithi153 ceph-mon[35045]: pgmap v597: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-09-11T22:22:23.513 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T22:22:23.513 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:23.513 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 513, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 490, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 11233188, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 206904, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.051762696, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.015414990, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.033604657, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006988880, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 489, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.017802745, 2024-09-11T22:22:23.514 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000036406 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 621040, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 110544 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 534, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 668, 2024-09-11T22:22:23.515 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 29496744, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 470043, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.060426959, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.014801971, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.029677000, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.011675109, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 666, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.019852184, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000029808 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.516 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 300544 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1150, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1150, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 21314625, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 364934, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-09-11T22:22:23.517 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.100665625, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.029648011, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.034856464, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.028155528, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1149, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.041007978, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000035690 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3417, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.072795124, 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000021303 2024-09-11T22:22:23.519 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-09-11T22:22:23.520 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-11T22:22:23.521 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 1870, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 1870, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 1870, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 2, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 1, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:23.522 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-11T22:22:23.523 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:23.524 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:23.525 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:23.526 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-11T22:22:23.527 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 59126, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 71, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 6, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 65060846, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 65059708, 2024-09-11T22:22:23.528 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 59126 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 13, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 2924, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 2911, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 11, 2024-09-11T22:22:23.529 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 174, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 163, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 1870, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 6493, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 4623, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 54344, 2024-09-11T22:22:23.530 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 215328 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T22:22:23.531 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.532 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:23.534 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.536 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:23.537 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.538 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.539 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.541 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2024-09-11T22:22:23.542 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1677, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:23.543 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:23.544 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T22:22:23.545 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 215191, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 66, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:23.546 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 166, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 1360072, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 7608, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T22:22:23.547 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 1982, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 1982, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1982, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.024980843, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000517144 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1982, 2024-09-11T22:22:23.548 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1983 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 1982, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 1982, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 1066, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 914, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T22:22:23.549 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T22:22:23.550 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T22:22:23.551 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 94, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 8, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 8, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-09-11T22:22:23.552 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:23.553 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 2193, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 61833565, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:23.554 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 2193, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2193, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 61833565, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:23.555 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1982, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 60869031, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1982, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 60869031, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1982, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1982, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1982, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1982, 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x5588b7978b40": { 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T22:22:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T22:22:23.561 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T22:22:24.253 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T22:22:24.253 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T22:22:24.253 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-11T22:22:24.253 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T22:22:24.253 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T22:22:24.515 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:22:25.173 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:24 smithi153 ceph-mon[35045]: pgmap v598: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 23 op/s 2024-09-11T22:22:25.175 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-11T22:22:25.175 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:25.175 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 275612, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 89886, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 93190549, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 64095798, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 13, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 8.366959687, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 3.141685736, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 7.246952027, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.117553704, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 89885, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.716573130, 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000019097 2024-09-11T22:22:25.176 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 353586, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.039174558, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000110 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3872, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 1184 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 4229, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 4363, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1083020, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 34151476, 2024-09-11T22:22:25.177 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 15, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 8, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.393679934, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.176781003, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.074167327, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.117393546, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4359, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.229995905, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000052763 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000281, 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000070 2024-09-11T22:22:25.178 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 301808 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 31304, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 30470, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 14803927, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 14436085, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 7, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 1.866408306, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.877643576, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 1.208117516, 2024-09-11T22:22:25.179 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.033885798, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 30471, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.658065377, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000021596 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 58437, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.009721727, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000166 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 644608, 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 114592 2024-09-11T22:22:25.180 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8475, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 4.487948346, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000529551 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:25.181 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4141, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.318645159, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000076948 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 70096, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 65975, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 65975, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 159.224209455, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002413402 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.182 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 4121, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 92, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 9, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 17, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 1, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 2, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 14, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 15, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 3159, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 1, 2024-09-11T22:22:25.183 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 1, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 4, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 147095, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 60911, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 4119, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 345, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 4, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 108, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 1891, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 45, 2024-09-11T22:22:25.184 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 2572, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 4, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 234, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:25.185 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 16, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 1832, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 4459, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 199455, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 1222, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 167, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 8169 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:25.186 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 370, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 349, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 1860, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 1860, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-11T22:22:25.187 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 111, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 9, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 59126, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-11T22:22:25.188 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 59126, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 71, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 71, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 6, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 65060846, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-11T22:22:25.189 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4042, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 9.930032058, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002456712 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 28, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 2976, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 2948, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 51, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 240, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 189, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 153, 2024-09-11T22:22:25.190 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 7453, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 7300, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 1, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 2942, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 2941, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 78120, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 223520 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 80211, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 2784, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 70096, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 221, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 6739, 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-11T22:22:25.191 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1219, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 46.686748632, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.038299219 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.001848469, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000924234 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.192 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 61304, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 19.342955561, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000315525 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.193 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.194 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.195 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 505, 2024-09-11T22:22:25.196 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.223172740, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000441926 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 104, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.060922022, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000585788 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1199, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 90.432106272, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.075422941 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:25.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.198 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-11T22:22:25.201 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1642, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2.476455759, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001508194 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-11T22:22:25.202 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 2, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 1, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 9874, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 9874, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 485, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:25.203 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:25.204 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:25.205 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 339561, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 129, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:25.206 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 166, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 256149, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 2425, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.207 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 8801, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 62491812, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 33, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 8801, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8801, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 25.378528950, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002883596 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-11T22:22:25.208 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8801, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 9493 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 8801, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 206, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 8595, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 206, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 4248, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 165, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-11T22:22:25.209 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 9, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 3746, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-11T22:22:25.211 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 94, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 4, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 12, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 12, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 789, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 300 2024-09-11T22:22:25.212 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 5, 2024-09-11T22:22:25.213 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 3720, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 1860, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 311141, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 81059464, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 311141, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:25.214 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 311141, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 81059464, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 8801, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 62694552, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 8801, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 62694552, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.216 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 8801, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 8801, 2024-09-11T22:22:25.217 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 8801, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 8801, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 1860, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 202740, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 1860, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 206, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 202740, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x55eccfde0960": { 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 59126, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 60866542, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 59126, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4042, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 60866542, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:25.221 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-11T22:22:25.221 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:24 smithi110 ceph-mon[34830]: pgmap v598: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 23 op/s 2024-09-11T22:22:25.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:24 smithi092 ceph-mon[31173]: pgmap v598: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 23 op/s 2024-09-11T22:22:26.128 INFO:tasks.check_counter:Daemon mds.c mds.exported=108 2024-09-11T22:22:26.128 INFO:tasks.check_counter:Daemon mds.c mds.imported=45 2024-09-11T22:22:26.128 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=9 2024-09-11T22:22:26.128 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T22:22:26.128 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T22:22:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:22:26.959 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-11T22:22:26.959 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:26.959 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 11606, 2024-09-11T22:22:26.959 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 12023, 2024-09-11T22:22:26.959 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 4832047, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 32058991, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 11, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.767759658, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.320358812, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.344834824, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.128312787, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 12021, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.393878321, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000032765 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.960 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 13928, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.002516227, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000180 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 240324, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 91670, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 80921523, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 44195110, 2024-09-11T22:22:26.961 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 11, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 7.951860075, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 3.016926569, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 6.802514020, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.075454404, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 91669, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.907638930, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000020810 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 320332, 2024-09-11T22:22:26.962 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.037287516, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000116 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 4688, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 302080 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 29199, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 29545, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 13941240, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 38847701, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 12, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:26.963 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 2.003069787, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.963137514, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 1.100035553, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.133578038, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 29544, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.906084784, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000030668 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 48590, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.008076674, 2024-09-11T22:22:26.964 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000166 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 632640, 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 114272 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:26.965 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8363, 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 5.455598463, 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000652349 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 7106, 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.740123466, 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000104154 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.966 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 66953, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 60957, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 60957, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 154.713920541, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002538082 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 5996, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 47, 2024-09-11T22:22:26.967 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 4, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 17, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 2, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 1, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 14, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 14, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 1491, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 4, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 138437, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 58972, 2024-09-11T22:22:26.968 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 5995, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 431, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 4, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 125, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 578, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 97, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 3942, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:26.969 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 4, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 234, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 37, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 1989, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 5528, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 164737, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 1056, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 223, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 10227 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.972 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 464, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 411, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 3244, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 3244, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:26.973 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 127, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 4, 2024-09-11T22:22:26.974 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 58677, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 58677, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 68, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-11T22:22:26.975 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 68, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 6, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 65765650, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4161, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 13.773469618, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003310134 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.976 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 28, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 3571, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 3543, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 48, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 245, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 197, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 272, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 9526, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 9254, 2024-09-11T22:22:26.977 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 3570, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 3570, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 81112, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 223520 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 78757, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 3030, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 66953, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 197, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 8502, 2024-09-11T22:22:26.978 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1053, 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 43.937507979, 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.041726028 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2, 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000635654, 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000317827 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.979 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 55127, 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 18.914098899, 2024-09-11T22:22:26.980 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000343100 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-11T22:22:26.981 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.982 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.983 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-11T22:22:26.983 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.984 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 588, 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.180932217, 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000307707 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.985 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 207, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.123589985, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000597053 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1087, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 86.709124721, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.079769203 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.986 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:26.987 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.988 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-11T22:22:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.990 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.990 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.990 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-11T22:22:26.990 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.990 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.990 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2893, 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 4.848031086, 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001675779 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.991 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 2, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 1, 2024-09-11T22:22:26.992 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 10193, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 10193, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 488, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:26.993 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:26.995 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 321948, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 118, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:26.996 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 166, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 348893, 2024-09-11T22:22:26.997 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 3935, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 11956, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 62706295, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 31, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 11956, 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-11T22:22:26.998 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 11956, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 37.354306569, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003124314 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 11956, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 12638 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 11956, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 322, 2024-09-11T22:22:26.999 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 11634, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 322, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 4483, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 238, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 4, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 6514, 2024-09-11T22:22:27.000 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:27.001 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-11T22:22:27.002 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 94, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 4, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 12, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 12, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 731, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-09-11T22:22:27.003 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 316 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:27.004 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 5, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 6488, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 3244, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 281125, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 74378403, 2024-09-11T22:22:27.005 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 281125, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 281125, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 74378403, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:27.006 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 11956, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 63059891, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 11956, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 63059891, 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:27.007 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:27.008 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 11956, 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 11956, 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 11956, 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 11956, 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:27.009 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 3244, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 353596, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 3244, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 322, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 353596, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:27.010 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x56303d5dc960": { 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 58677, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 61571346, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 58677, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:27.011 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4161, 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 61571346, 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:27.012 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-11T22:22:27.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:26 smithi110 ceph-mon[34830]: pgmap v599: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-11T22:22:27.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:26 smithi092 ceph-mon[31173]: pgmap v599: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-11T22:22:27.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:26 smithi153 ceph-mon[35045]: pgmap v599: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-11T22:22:27.813 INFO:tasks.check_counter:Daemon mds.f mds.exported=125 2024-09-11T22:22:27.814 INFO:tasks.check_counter:Daemon mds.f mds.imported=97 2024-09-11T22:22:27.814 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=4 2024-09-11T22:22:27.814 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T22:22:27.814 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T22:22:28.045 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:27 smithi153 ceph-mon[35045]: pgmap v600: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-09-11T22:22:28.064 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:22:28.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:27 smithi092 ceph-mon[31173]: pgmap v600: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-09-11T22:22:28.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:27 smithi110 ceph-mon[34830]: pgmap v600: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-09-11T22:22:28.631 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-11T22:22:28.631 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 410904, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 257096, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 136324773, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 126067164, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 15, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 14.324031633, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 5.542994057, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 11.652379350, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.096195731, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 257095, 2024-09-11T22:22:28.632 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 3.707175030, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000014419 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 649672, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.085539186, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000131 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 648320, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 113952 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 56047, 2024-09-11T22:22:28.633 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 54029, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 16642327, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 69485088, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 10, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 7, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 3.150510645, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 1.391644980, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 2.004130944, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.148570776, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 54028, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.069169479, 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000019789 2024-09-11T22:22:28.634 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 97747, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.015370281, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000157 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 302080 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 33970, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 34390, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 9572832, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 83537371, 2024-09-11T22:22:28.635 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 9, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 2.171165724, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.975151657, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 1.103044415, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.247243912, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 34388, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.720266787, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000020945 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 47451, 2024-09-11T22:22:28.636 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.007665485, 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000161 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:28.637 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 28512, 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 26.511121421, 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000929823 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3914, 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.364960763, 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000093244 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.638 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 166535, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 72247, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 72247, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 514.461774874, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.007120873 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 94288, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 218, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 23, 2024-09-11T22:22:28.639 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 927, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 226, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 688, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 13, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 584, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 14825, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 2, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 2, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 6, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 227789, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 96043, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 93533, 2024-09-11T22:22:28.640 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 341, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 1000, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 1, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 180, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 10270, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 418, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 1876, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:28.641 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 4, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 234, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 4, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 20508, 2024-09-11T22:22:28.642 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 12467, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 1, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 214750, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 7326, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 198, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 32046 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 342, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 523, 2024-09-11T22:22:28.643 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 18, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 1630, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 1612, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:28.644 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 200, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 23, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 87263, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 87263, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-11T22:22:28.645 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 103, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 103, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 9, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 141672388, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 14630, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 34.075986406, 2024-09-11T22:22:28.646 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002329185 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 54, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 7801, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 7747, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 179, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 661, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 482, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 898, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 19471, 2024-09-11T22:22:28.647 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 18573, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 2, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 7708, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 7706, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 87692, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 223520 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 182509, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 22081, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 166535, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 333, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 42026, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 7338, 2024-09-11T22:22:28.648 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 22.090849310, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003010472 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 383, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.096392441, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000251677 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-11T22:22:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 54214, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 344.177041320, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.006348490 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.650 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.651 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-11T22:22:28.652 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 73, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.127984362, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001753210 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1764, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.050718474, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000595645 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 104, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.074851295, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000719723 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 7293, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 144.583369930, 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.019824951 2024-09-11T22:22:28.654 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 75, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.939010703, 2024-09-11T22:22:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.012520142 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.656 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-11T22:22:28.657 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.658 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1003, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.321557039, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001317604 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.659 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 1, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 0, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 6684, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 6684, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 486, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:28.660 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:28.661 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:28.662 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 297861, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 114, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-11T22:22:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 166, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 1063731, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 9237, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 19484, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 141795151, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 33, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 19484, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 19484, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 45.574154745, 2024-09-11T22:22:28.665 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002339055 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 19484, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 20390 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 19484, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 258, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 19226, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2024-09-11T22:22:28.666 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 258, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 14888, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 208, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 23, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 3523, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 0, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2024-09-11T22:22:28.667 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 1, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-11T22:22:28.668 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 94, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-11T22:22:28.669 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 4, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 12, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 12, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 1113, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 346 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.670 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 17, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 5, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 3479, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 1612, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:28.671 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 500917, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 117442192, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 500917, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 500917, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 117442192, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.672 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 19484, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 141970715, 2024-09-11T22:22:28.673 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 19484, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 141970715, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:28.674 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 19484, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 19484, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 19484, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 19484, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:28.675 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 1612, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 175564, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 1612, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 258, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 175564, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.676 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x5570a3f04960": { 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 87263, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 137478084, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 87263, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 14630, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 137478084, 2024-09-11T22:22:28.677 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:28.678 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:28.678 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:28.678 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:28.678 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.678 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:28.678 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-11T22:22:29.389 INFO:tasks.check_counter:Daemon mds.i mds.exported=180 2024-09-11T22:22:29.389 INFO:tasks.check_counter:Daemon mds.i mds.imported=418 2024-09-11T22:22:29.389 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=23 2024-09-11T22:22:29.389 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T22:22:29.390 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T22:22:29.636 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1294, 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1296, 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 53971629, 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 411499, 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 8, 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-11T22:22:30.194 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.131795396, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.031280793, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.057227400, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.033774617, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1294, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.042094440, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000032530 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 768, 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 745, 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 51094584, 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 288694, 2024-09-11T22:22:30.196 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.091025495, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.020388963, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.061292821, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.011863956, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 744, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.023559098, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000031665 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:30.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 621584, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 111824 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 260, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 394, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 33622560, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 384050, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2024-09-11T22:22:30.198 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.041798198, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.009990496, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.024415944, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.004975120, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 393, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.016391983, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000041709 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.199 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 302960 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3602, 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.078428947, 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000021773 2024-09-11T22:22:30.200 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-11T22:22:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 3632, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 130, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 3502, 2024-09-11T22:22:30.202 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 3446, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 1, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-11T22:22:30.203 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-11T22:22:30.204 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-11T22:22:30.205 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-11T22:22:30.206 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-11T22:22:30.207 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 87263, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 103, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 9, 2024-09-11T22:22:30.208 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 141672388, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 141670737, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 87263 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 31, 2024-09-11T22:22:30.209 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 7607, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 7576, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 155, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 555, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 400, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 3632, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 17111, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 13479, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 61536, 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 215328 2024-09-11T22:22:30.210 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.211 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.212 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.213 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.214 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.215 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.215 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.215 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-11T22:22:30.215 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.215 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.215 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.215 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.216 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.217 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.219 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.220 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-11T22:22:30.221 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.222 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 1, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 0, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 1706, 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-11T22:22:30.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:22:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:22:30.225 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 239426, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 79, 2024-09-11T22:22:30.226 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 8856, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 166, 2024-09-11T22:22:30.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 2840335, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 25893, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-11T22:22:30.228 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 2105, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 21, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 3, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 2105, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2105, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.378974431, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000655094 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-11T22:22:30.229 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2105, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2106 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 2105, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 2105, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 1112, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 991, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-11T22:22:30.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 1, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-11T22:22:30.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-11T22:22:30.232 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 94, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 4, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 11, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 11, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-09-11T22:22:30.233 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-11T22:22:30.234 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 2318, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 138466427, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 2318, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:30.235 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2318, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 138466427, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 2105, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 137479983, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2105, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 137479983, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-11T22:22:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 2105, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 2105, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2105, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 2105, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.238 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:30.239 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x560a77dfcb40": { 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-11T22:22:30.240 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-11T22:22:30.241 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-11T22:22:30.242 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.242 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-11T22:22:30.242 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-11T22:22:31.022 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:30 smithi153 ceph-mon[35045]: pgmap v601: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-11T22:22:31.048 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T22:22:31.048 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T22:22:31.048 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-11T22:22:31.049 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-11T22:22:31.058 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-11T22:22:31.059 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:22:31.059 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:22:31.079 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:22:31.080 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:22:31.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T22:22:31.206 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:30 smithi092 ceph-mon[31173]: pgmap v601: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-11T22:22:31.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:30 smithi110 ceph-mon[34830]: pgmap v601: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-11T22:22:31.445 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:31.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 -- 172.21.15.92:0/4278866687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f1010c180 msgr2=0x7f2f1010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:31.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/4278866687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f1010c180 0x7f2f1010e600 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009ca0 tx=0x7f2f0802a950 comp rx=0 tx=0).stop 2024-09-11T22:22:31.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 -- 172.21.15.92:0/4278866687 shutdown_connections 2024-09-11T22:22:31.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/4278866687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f1010c180 0x7f2f1010e600 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/4278866687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f101036c0 0x7f2f1010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/4278866687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f10102da0 0x7f2f10103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 -- 172.21.15.92:0/4278866687 >> 172.21.15.92:0/4278866687 conn(0x7f2f100fc7a0 msgr2=0x7f2f100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 -- 172.21.15.92:0/4278866687 shutdown_connections 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.907+0000 7f2f16b8b640 1 -- 172.21.15.92:0/4278866687 wait complete. 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 Processor -- start 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 -- start start 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f10102da0 0x7f2f10100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f101036c0 0x7f2f10100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f1010c180 0x7f2f10101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10111dd0 con 0x7f2f10102da0 2024-09-11T22:22:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10111c50 con 0x7f2f1010c180 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f16b8b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10111f50 con 0x7f2f101036c0 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f14927640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f10102da0 0x7f2f10100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f14927640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f10102da0 0x7f2f10100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35258/0 (socket says 172.21.15.92:35258) 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f14927640 1 -- 172.21.15.92:0/1321533132 learned_addr learned my addr 172.21.15.92:0/1321533132 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f15128640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f1010c180 0x7f2f10101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.908+0000 7f2f0ffff640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f101036c0 0x7f2f10100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f14927640 1 -- 172.21.15.92:0/1321533132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f101036c0 msgr2=0x7f2f10100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f14927640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f101036c0 0x7f2f10100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f14927640 1 -- 172.21.15.92:0/1321533132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f1010c180 msgr2=0x7f2f10101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f14927640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f1010c180 0x7f2f10101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f14927640 1 -- 172.21.15.92:0/1321533132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f101a67c0 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f0ffff640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f101036c0 0x7f2f10100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f14927640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f10102da0 0x7f2f10100a00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400d950 tx=0x7f2f0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f0dffb640 1 -- 172.21.15.92:0/1321533132 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04017070 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f0dffb640 1 -- 172.21.15.92:0/1321533132 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f04009c30 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f0dffb640 1 -- 172.21.15.92:0/1321533132 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400f620 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f101a6ab0 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.909+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f101a7040 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.910+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f10103180 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.911+0000 7f2f0dffb640 1 -- 172.21.15.92:0/1321533132 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2f040109a0 con 0x7f2f10102da0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.912+0000 7f2f0dffb640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2f000a2c00 0x7f2f000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.912+0000 7f2f0ffff640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2f000a2c00 0x7f2f000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:31.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.912+0000 7f2f0dffb640 1 -- 172.21.15.92:0/1321533132 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2f040cef90 con 0x7f2f10102da0 2024-09-11T22:22:31.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.913+0000 7f2f0ffff640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2f000a2c00 0x7f2f000a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f10102490 tx=0x7f2efc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:31.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:31.917+0000 7f2f0dffb640 1 -- 172.21.15.92:0/1321533132 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f04009de0 con 0x7f2f10102da0 2024-09-11T22:22:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.096+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2f10101f30 con 0x7f2f10102da0 2024-09-11T22:22:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.096+0000 7f2f0dffb640 1 -- 172.21.15.92:0/1321533132 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+589 (secure 0 0 0) 0x7f2f0409bb60 con 0x7f2f10102da0 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2458669467 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2872938890 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3563316301 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3468435378 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/125981315 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:32.097 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/4186575934 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:32.098 INFO:teuthology.orchestra.run.smithi092.stderr:listed 10 entries 2024-09-11T22:22:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.098+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2f000a2c00 msgr2=0x7f2f000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.098+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2f000a2c00 0x7f2f000a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f10102490 tx=0x7f2efc005f90 comp rx=0 tx=0).stop 2024-09-11T22:22:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f10102da0 msgr2=0x7f2f10100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f10102da0 0x7f2f10100a00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400d950 tx=0x7f2f0400de10 comp rx=0 tx=0).stop 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 shutdown_connections 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2f000a2c00 0x7f2f000a50c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f1010c180 0x7f2f10101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f101036c0 0x7f2f10100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 --2- 172.21.15.92:0/1321533132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f10102da0 0x7f2f10100a00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 >> 172.21.15.92:0/1321533132 conn(0x7f2f100fc7a0 msgr2=0x7f2f100fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 shutdown_connections 2024-09-11T22:22:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:32.099+0000 7f2f16b8b640 1 -- 172.21.15.92:0/1321533132 wait complete. 2024-09-11T22:22:32.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:32 smithi092 ceph-mon[31173]: pgmap v602: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:32.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:32 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/1321533132' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:22:32.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:32 smithi153 ceph-mon[35045]: pgmap v602: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:32.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:32 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/1321533132' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:22:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:32 smithi110 ceph-mon[34830]: pgmap v602: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:32.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:32 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/1321533132' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:22:32.734 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T22:22:32.734 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:22:32.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T22:22:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:33.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 -- 172.21.15.92:0/4027976976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04105ad0 msgr2=0x7f4f04105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:33.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 --2- 172.21.15.92:0/4027976976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04105ad0 0x7f4f04105eb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009900 tx=0x7f4ef402a410 comp rx=0 tx=0).stop 2024-09-11T22:22:33.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 -- 172.21.15.92:0/4027976976 shutdown_connections 2024-09-11T22:22:33.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 --2- 172.21.15.92:0/4027976976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f04101ba0 0x7f4f0410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 --2- 172.21.15.92:0/4027976976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f04101200 0x7f4f04101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 --2- 172.21.15.92:0/4027976976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04105ad0 0x7f4f04105eb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 -- 172.21.15.92:0/4027976976 >> 172.21.15.92:0/4027976976 conn(0x7f4f04077210 msgr2=0x7f4f040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 -- 172.21.15.92:0/4027976976 shutdown_connections 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.574+0000 7f4f0a853640 1 -- 172.21.15.92:0/4027976976 wait complete. 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 Processor -- start 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 -- start start 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04101200 0x7f4f040fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f04101ba0 0x7f4f040fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:33.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f04105ad0 0x7f4f040fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:33.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04110da0 con 0x7f4f04101200 2024-09-11T22:22:33.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04110c20 con 0x7f4f04101ba0 2024-09-11T22:22:33.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.575+0000 7f4f0a853640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04110f20 con 0x7f4f04105ad0 2024-09-11T22:22:33.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04101200 0x7f4f040fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:33.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04101200 0x7f4f040fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35276/0 (socket says 172.21.15.92:35276) 2024-09-11T22:22:33.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 -- 172.21.15.92:0/3602171373 learned_addr learned my addr 172.21.15.92:0/3602171373 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:33.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f08df0640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f04105ad0 0x7f4f040fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f037fe640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f04101ba0 0x7f4f040fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 -- 172.21.15.92:0/3602171373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f04105ad0 msgr2=0x7f4f040fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f04105ad0 0x7f4f040fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 -- 172.21.15.92:0/3602171373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f04101ba0 msgr2=0x7f4f040fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f04101ba0 0x7f4f040fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 -- 172.21.15.92:0/3602171373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f041a67f0 con 0x7f4f04101200 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f037fe640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f04101ba0 0x7f4f040fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f03fff640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04101200 0x7f4f040fc690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4006010 tx=0x7f4ef40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f017fa640 1 -- 172.21.15.92:0/3602171373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef403f070 con 0x7f4f04101200 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f017fa640 1 -- 172.21.15.92:0/3602171373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef4003aa0 con 0x7f4f04101200 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f017fa640 1 -- 172.21.15.92:0/3602171373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4035030 con 0x7f4f04101200 2024-09-11T22:22:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f041a6a80 con 0x7f4f04101200 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.576+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041a6f60 con 0x7f4f04101200 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.577+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f04101660 con 0x7f4f04101200 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.578+0000 7f4f017fa640 1 -- 172.21.15.92:0/3602171373 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ef4042070 con 0x7f4f04101200 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.579+0000 7f4f017fa640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f4eec0a2b80 0x7f4eec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.579+0000 7f4f037fe640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f4eec0a2b80 0x7f4eec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.579+0000 7f4f017fa640 1 -- 172.21.15.92:0/3602171373 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4ef40efdd0 con 0x7f4f04101200 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.579+0000 7f4f037fe640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f4eec0a2b80 0x7f4eec0a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4f040fe120 tx=0x7f4ef8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.583+0000 7f4f017fa640 1 -- 172.21.15.92:0/3602171373 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ef40bc9a0 con 0x7f4f04101200 2024-09-11T22:22:33.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.762+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f0402d7c0 con 0x7f4f04101200 2024-09-11T22:22:33.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.762+0000 7f4f017fa640 1 -- 172.21.15.92:0/3602171373 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+589 (secure 0 0 0) 0x7f4ef40bc2a0 con 0x7f4f04101200 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2458669467 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2872938890 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3563316301 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3468435378 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/125981315 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/1519869556 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/733069068 2024-09-12T22:05:15.603416+0000 2024-09-11T22:22:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/4186575934 2024-09-12T22:04:52.791314+0000 2024-09-11T22:22:33.764 INFO:teuthology.orchestra.run.smithi092.stderr:listed 10 entries 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.764+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f4eec0a2b80 msgr2=0x7f4eec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.764+0000 7f4f0a853640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f4eec0a2b80 0x7f4eec0a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4f040fe120 tx=0x7f4ef8005d30 comp rx=0 tx=0).stop 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.764+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04101200 msgr2=0x7f4f040fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.764+0000 7f4f0a853640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04101200 0x7f4f040fc690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4006010 tx=0x7f4ef40042f0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.764+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 shutdown_connections 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.765+0000 7f4f0a853640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f4eec0a2b80 0x7f4eec0a5040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.765+0000 7f4f0a853640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f04105ad0 0x7f4f040fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.765+0000 7f4f0a853640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f04101ba0 0x7f4f040fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.765+0000 7f4f0a853640 1 --2- 172.21.15.92:0/3602171373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f04101200 0x7f4f040fc690 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:33.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.765+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 >> 172.21.15.92:0/3602171373 conn(0x7f4f04077210 msgr2=0x7f4f041006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:33.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.765+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 shutdown_connections 2024-09-11T22:22:33.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:33.765+0000 7f4f0a853640 1 -- 172.21.15.92:0/3602171373 wait complete. 2024-09-11T22:22:34.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:33 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3602171373' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:22:34.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:33 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3602171373' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:22:34.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:33 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3602171373' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:22:34.428 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2024-09-11T22:22:34.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:34.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:34.455 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T22:22:34.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[86294]: fuse finished with error 0 and tester_r 0 2024-09-11T22:22:35.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:34 smithi110 ceph-mon[34830]: pgmap v603: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:22:35.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:34 smithi092 ceph-mon[31173]: pgmap v603: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:22:35.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:34 smithi153 ceph-mon[35045]: pgmap v603: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:22:37.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:36 smithi110 ceph-mon[34830]: pgmap v604: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:22:37.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:36 smithi092 ceph-mon[31173]: pgmap v604: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:22:37.288 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:36 smithi153 ceph-mon[35045]: pgmap v604: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:22:39.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:38 smithi110 ceph-mon[34830]: pgmap v605: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:39.268 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:38 smithi092 ceph-mon[31173]: pgmap v605: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:39.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:38 smithi153 ceph-mon[35045]: pgmap v605: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:40.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T22:22:40.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:40.557 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T22:22:40.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:40.576 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-11T22:22:40.634 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T22:22:40.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:40.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:22:40.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:40.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-09-11T22:22:40.804 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T22:22:40.817 INFO:tasks.cephadm:Teardown begin 2024-09-11T22:22:40.817 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T22:22:40.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:22:41.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:40 smithi092 ceph-mon[31173]: pgmap v606: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T22:22:41.069 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:41.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:40 smithi110 ceph-mon[34830]: pgmap v606: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T22:22:41.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:40 smithi153 ceph-mon[35045]: pgmap v606: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T22:22:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 -- 172.21.15.92:0/3190572642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 msgr2=0x7f2000100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 --2- 172.21.15.92:0/3190572642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000100ed0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009930 tx=0x7f1ff002a420 comp rx=0 tx=0).stop 2024-09-11T22:22:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 -- 172.21.15.92:0/3190572642 shutdown_connections 2024-09-11T22:22:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 --2- 172.21.15.92:0/3190572642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000101410 0x7f200010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 --2- 172.21.15.92:0/3190572642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000100ed0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 --2- 172.21.15.92:0/3190572642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2000106a70 0x7f2000106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 -- 172.21.15.92:0/3190572642 >> 172.21.15.92:0/3190572642 conn(0x7f20000fc790 msgr2=0x7f20000febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 -- 172.21.15.92:0/3190572642 shutdown_connections 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 -- 172.21.15.92:0/3190572642 wait complete. 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.593+0000 7f20075f6640 1 Processor -- start 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f20075f6640 1 -- start start 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f20075f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f20075f6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2000101410 0x7f20001988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f20075f6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000106a70 0x7f200019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f20075f6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110d40 con 0x7f2000100a70 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f20075f6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110bc0 con 0x7f2000101410 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f20075f6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110ec0 con 0x7f2000106a70 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47180/0 (socket says 172.21.15.92:47180) 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 -- 172.21.15.92:0/118934632 learned_addr learned my addr 172.21.15.92:0/118934632 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005b93640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000106a70 0x7f200019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 -- 172.21.15.92:0/118934632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000106a70 msgr2=0x7f200019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000106a70 0x7f200019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 -- 172.21.15.92:0/118934632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2000101410 msgr2=0x7f20001988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2000101410 0x7f20001988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 -- 172.21.15.92:0/118934632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200019d330 con 0x7f2000100a70 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f2005392640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000198380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c920 tx=0x7f1ffc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.594+0000 7f1ff67fc640 1 -- 172.21.15.92:0/118934632 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc007c70 con 0x7f2000100a70 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.595+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200019d620 con 0x7f2000100a70 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.595+0000 7f1ff67fc640 1 -- 172.21.15.92:0/118934632 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ffc007e10 con 0x7f2000100a70 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.595+0000 7f1ff67fc640 1 -- 172.21.15.92:0/118934632 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc018820 con 0x7f2000100a70 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.595+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001a4f00 con 0x7f2000100a70 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.596+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2000100ed0 con 0x7f2000100a70 2024-09-11T22:22:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.597+0000 7f1ff67fc640 1 -- 172.21.15.92:0/118934632 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ffc0189c0 con 0x7f2000100a70 2024-09-11T22:22:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.597+0000 7f1ff67fc640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1fec0a2b80 0x7f1fec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.597+0000 7f2004b91640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1fec0a2b80 0x7f1fec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.597+0000 7f1ff67fc640 1 -- 172.21.15.92:0/118934632 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ffc0cf5f0 con 0x7f2000100a70 2024-09-11T22:22:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.598+0000 7f2004b91640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1fec0a2b80 0x7f1fec0a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00096c0 tx=0x7f1ff0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.601+0000 7f1ff67fc640 1 -- 172.21.15.92:0/118934632 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ffc014030 con 0x7f2000100a70 2024-09-11T22:22:41.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.804+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f200010a070 con 0x7f2000100a70 2024-09-11T22:22:41.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.805+0000 7f1ff67fc640 1 -- 172.21.15.92:0/118934632 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+12015 (secure 0 0 0) 0x7f1ffc09c1c0 con 0x7f2000100a70 2024-09-11T22:22:41.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:22:41.807 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":47,"btime":"2024-09-11T22:21:42:892765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:21:42.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:22:41.807 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 47 2024-09-11T22:22:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.807+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1fec0a2b80 msgr2=0x7f1fec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.807+0000 7f20075f6640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1fec0a2b80 0x7f1fec0a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00096c0 tx=0x7f1ff0031400 comp rx=0 tx=0).stop 2024-09-11T22:22:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.807+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 msgr2=0x7f2000198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.807+0000 7f20075f6640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000198380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c920 tx=0x7f1ffc00cde0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 shutdown_connections 2024-09-11T22:22:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f1fec0a2b80 0x7f1fec0a5040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2000106a70 0x7f200019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2000101410 0x7f20001988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 --2- 172.21.15.92:0/118934632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2000100a70 0x7f2000198380 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 >> 172.21.15.92:0/118934632 conn(0x7f20000fc790 msgr2=0x7f200006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 shutdown_connections 2024-09-11T22:22:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:41.808+0000 7f20075f6640 1 -- 172.21.15.92:0/118934632 wait complete. 2024-09-11T22:22:42.110 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:41 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/118934632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:42.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:41 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/118934632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:42.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:41 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/118934632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:42.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:22:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:43.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:42 smithi092 ceph-mon[31173]: pgmap v607: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:22:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 -- 172.21.15.92:0/936464648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 msgr2=0x7fd4c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 --2- 172.21.15.92:0/936464648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c810da40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009ca0 tx=0x7fd4c002a950 comp rx=0 tx=0).stop 2024-09-11T22:22:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 -- 172.21.15.92:0/936464648 shutdown_connections 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 --2- 172.21.15.92:0/936464648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c810da40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 --2- 172.21.15.92:0/936464648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4c8102570 0x7fd4c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 --2- 172.21.15.92:0/936464648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8100e20 0x7fd4c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 -- 172.21.15.92:0/936464648 >> 172.21.15.92:0/936464648 conn(0x7fd4c80fc7d0 msgr2=0x7fd4c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 -- 172.21.15.92:0/936464648 shutdown_connections 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.148+0000 7fd4d027c640 1 -- 172.21.15.92:0/936464648 wait complete. 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 Processor -- start 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 -- start start 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8100e20 0x7fd4c8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4c8102570 0x7fd4c8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8110ed0 con 0x7fd4c8102ef0 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8110d50 con 0x7fd4c8100e20 2024-09-11T22:22:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4d027c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8111050 con 0x7fd4c8102570 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4ce819640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4ce819640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47190/0 (socket says 172.21.15.92:47190) 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4ce819640 1 -- 172.21.15.92:0/2836166751 learned_addr learned my addr 172.21.15.92:0/2836166751 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4cd817640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4c8102570 0x7fd4c8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.149+0000 7fd4ce018640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8100e20 0x7fd4c8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4ce819640 1 -- 172.21.15.92:0/2836166751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4c8102570 msgr2=0x7fd4c8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4ce819640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4c8102570 0x7fd4c8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4ce819640 1 -- 172.21.15.92:0/2836166751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8100e20 msgr2=0x7fd4c8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4ce819640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8100e20 0x7fd4c8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4ce819640 1 -- 172.21.15.92:0/2836166751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c819d4d0 con 0x7fd4c8102ef0 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4ce819640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c819cdf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006010 tx=0x7fd4c002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4bf7fe640 1 -- 172.21.15.92:0/2836166751 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0034040 con 0x7fd4c8102ef0 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4bf7fe640 1 -- 172.21.15.92:0/2836166751 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4c0003e00 con 0x7fd4c8102ef0 2024-09-11T22:22:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c819d760 con 0x7fd4c8102ef0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4bf7fe640 1 -- 172.21.15.92:0/2836166751 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0045480 con 0x7fd4c8102ef0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.150+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c81a50a0 con 0x7fd4c8102ef0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.151+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c8101280 con 0x7fd4c8102ef0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.152+0000 7fd4bf7fe640 1 -- 172.21.15.92:0/2836166751 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4c0037020 con 0x7fd4c8102ef0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.153+0000 7fd4bf7fe640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd4b40a2b80 0x7fd4b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.153+0000 7fd4ce018640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd4b40a2b80 0x7fd4b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.153+0000 7fd4bf7fe640 1 -- 172.21.15.92:0/2836166751 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd4c00f03f0 con 0x7fd4c8102ef0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.154+0000 7fd4ce018640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd4b40a2b80 0x7fd4b40a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009990 tx=0x7fd4c4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.157+0000 7fd4bf7fe640 1 -- 172.21.15.92:0/2836166751 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4c00bcfc0 con 0x7fd4c8102ef0 2024-09-11T22:22:43.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:42 smithi110 ceph-mon[34830]: pgmap v607: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:22:43.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:42 smithi153 ceph-mon[35045]: pgmap v607: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:22:43.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.361+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4c81a5270 con 0x7fd4c8102ef0 2024-09-11T22:22:43.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.362+0000 7fd4bf7fe640 1 -- 172.21.15.92:0/2836166751 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+12015 (secure 0 0 0) 0x7fd4c00bc8c0 con 0x7fd4c8102ef0 2024-09-11T22:22:43.363 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:22:43.363 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":47,"btime":"2024-09-11T22:21:42:892765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1683671615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1683671615},{"type":"v1","addr":"172.21.15.92:6835","nonce":1683671615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2477546203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2477546203},{"type":"v1","addr":"172.21.15.92:6837","nonce":2477546203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:12:03.420066+0000","modified":"2024-09-11T22:21:42.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24451,"mds_2":24433,"mds_3":24428,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1790524059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1790524059},{"type":"v1","addr":"172.21.15.92:6839","nonce":1790524059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6841/2477024139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2477024139},{"type":"v1","addr":"172.21.15.92:6841","nonce":2477024139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/797341789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":797341789},{"type":"v1","addr":"172.21.15.110:6833","nonce":797341789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/490387556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":490387556},{"type":"v1","addr":"172.21.15.153:6833","nonce":490387556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1455978865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1455978865},{"type":"v1","addr":"172.21.15.110:6835","nonce":1455978865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.153:6835/1638990868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1638990868},{"type":"v1","addr":"172.21.15.153:6835","nonce":1638990868}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.110:6837/3571280995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3571280995},{"type":"v1","addr":"172.21.15.110:6837","nonce":3571280995}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6837/1434479445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1434479445},{"type":"v1","addr":"172.21.15.153:6837","nonce":1434479445}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.153:6839/229095273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":229095273},{"type":"v1","addr":"172.21.15.153:6839","nonce":229095273}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/538708966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":538708966},{"type":"v1","addr":"172.21.15.110:6839","nonce":538708966}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24451,24433,24428,24427]},"id":1}]} 2024-09-11T22:22:43.363 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 47 2024-09-11T22:22:43.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd4b40a2b80 msgr2=0x7fd4b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd4b40a2b80 0x7fd4b40a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009990 tx=0x7fd4c4009110 comp rx=0 tx=0).stop 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 msgr2=0x7fd4c819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c819cdf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006010 tx=0x7fd4c002f710 comp rx=0 tx=0).stop 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 shutdown_connections 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fd4b40a2b80 0x7fd4b40a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4c8102ef0 0x7fd4c819cdf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4c8102570 0x7fd4c8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 --2- 172.21.15.92:0/2836166751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8100e20 0x7fd4c8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 >> 172.21.15.92:0/2836166751 conn(0x7fd4c80fc7d0 msgr2=0x7fd4c8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 shutdown_connections 2024-09-11T22:22:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:43.365+0000 7fd4d027c640 1 -- 172.21.15.92:0/2836166751 wait complete. 2024-09-11T22:22:44.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:43 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2836166751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:44.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:22:44.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:43 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2836166751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:44.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:43 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2836166751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:44.294 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:44.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.771+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3080747380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 msgr2=0x7f2ef410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:44.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.771+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3080747380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef410d940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009cd0 tx=0x7f2eec02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.771+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3080747380 shutdown_connections 2024-09-11T22:22:44.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.771+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3080747380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef410d940 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.771+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3080747380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4100a70 0x7f2ef4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.771+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3080747380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ef4106a70 0x7f2ef4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.771+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3080747380 >> 172.21.15.92:0/3080747380 conn(0x7f2ef40fc790 msgr2=0x7f2ef40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:44.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.772+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3080747380 shutdown_connections 2024-09-11T22:22:44.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.772+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3080747380 wait complete. 2024-09-11T22:22:44.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.772+0000 7f2efc1b7640 1 Processor -- start 2024-09-11T22:22:44.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.772+0000 7f2efc1b7640 1 -- start start 2024-09-11T22:22:44.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2efc1b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4100a70 0x7f2ef41973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:44.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2efc1b7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef4197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:44.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2efc1b7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ef4106a70 0x7f2ef419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:44.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2efc1b7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4110dc0 con 0x7f2ef4101410 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2efc1b7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4110c40 con 0x7f2ef4106a70 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2efc1b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4110f40 con 0x7f2ef4100a70 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef4197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef4197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47206/0 (socket says 172.21.15.92:47206) 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 -- 172.21.15.92:0/3434128293 learned_addr learned my addr 172.21.15.92:0/3434128293 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9f53640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4100a70 0x7f2ef41973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 -- 172.21.15.92:0/3434128293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4100a70 msgr2=0x7f2ef41973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2efa754640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ef4106a70 0x7f2ef419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4100a70 0x7f2ef41973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 -- 172.21.15.92:0/3434128293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ef4106a70 msgr2=0x7f2ef419bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ef4106a70 0x7f2ef419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 -- 172.21.15.92:0/3434128293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef419c390 con 0x7f2ef4101410 2024-09-11T22:22:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9f53640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4100a70 0x7f2ef41973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2ef9752640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef4197920 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400b4a0 tx=0x7f2ee400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.773+0000 7f2eeaffd640 1 -- 172.21.15.92:0/3434128293 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4014070 con 0x7f2ef4101410 2024-09-11T22:22:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.774+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef419c680 con 0x7f2ef4101410 2024-09-11T22:22:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.774+0000 7f2eeaffd640 1 -- 172.21.15.92:0/3434128293 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee4010ac0 con 0x7f2ef4101410 2024-09-11T22:22:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.774+0000 7f2eeaffd640 1 -- 172.21.15.92:0/3434128293 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4017c30 con 0x7f2ef4101410 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.774+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41a3f60 con 0x7f2ef4101410 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.774+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef4100f50 con 0x7f2ef4101410 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.776+0000 7f2eeaffd640 1 -- 172.21.15.92:0/3434128293 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ee4017070 con 0x7f2ef4101410 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.776+0000 7f2eeaffd640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2ee00a2b80 0x7f2ee00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.777+0000 7f2ef9f53640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2ee00a2b80 0x7f2ee00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.777+0000 7f2eeaffd640 1 -- 172.21.15.92:0/3434128293 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ee40ce8d0 con 0x7f2ef4101410 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.777+0000 7f2ef9f53640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2ee00a2b80 0x7f2ee00a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006fd0 tx=0x7f2ef0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.781+0000 7f2eeaffd640 1 -- 172.21.15.92:0/3434128293 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ee409b440 con 0x7f2ef4101410 2024-09-11T22:22:44.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.952+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ef4108a70 con 0x7f2ef4101410 2024-09-11T22:22:44.954 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:44 smithi092 ceph-mon[31173]: pgmap v608: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:22:44.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.954+0000 7f2eeaffd640 1 -- 172.21.15.92:0/3434128293 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24831 (secure 0 0 0) 0x7f2ee409ad40 con 0x7f2ef4101410 2024-09-11T22:22:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T22:22:44.956 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"fsid":"a6709e28-7089-11ef-bcea-c7b262605968","created":"2024-09-11T22:04:29.869322+0000","modified":"2024-09-11T22:14:17.777929+0000","last_up_change":"2024-09-11T22:11:13.770691+0000","last_in_change":"2024-09-11T22:10:51.231812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:07:29.899850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:11:59.645581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:12:01.559996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T22:12:08.023536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4595b0bf-4ae7-499d-98e9-3aa88e41ff70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6803","nonce":2928117998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6805","nonce":2928117998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6809","nonce":2928117998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2928117998},{"type":"v1","addr":"172.21.15.92:6807","nonce":2928117998}]},"public_addr":"172.21.15.92:6803/2928117998","cluster_addr":"172.21.15.92:6805/2928117998","heartbeat_back_addr":"172.21.15.92:6809/2928117998","heartbeat_front_addr":"172.21.15.92:6807/2928117998","state":["exists","up"]},{"osd":1,"uuid":"26c935bd-8f25-4bbd-bdcf-8d55523c3e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6801","nonce":1945048542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6803","nonce":1945048542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6807","nonce":1945048542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1945048542},{"type":"v1","addr":"172.21.15.110:6805","nonce":1945048542}]},"public_addr":"172.21.15.110:6801/1945048542","cluster_addr":"172.21.15.110:6803/1945048542","heartbeat_back_addr":"172.21.15.110:6807/1945048542","heartbeat_front_addr":"172.21.15.110:6805/1945048542","state":["exists","up"]},{"osd":2,"uuid":"1be9b69c-ef19-405e-9f42-0964e338d4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6801","nonce":2033986544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6803","nonce":2033986544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6807","nonce":2033986544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2033986544},{"type":"v1","addr":"172.21.15.153:6805","nonce":2033986544}]},"public_addr":"172.21.15.153:6801/2033986544","cluster_addr":"172.21.15.153:6803/2033986544","heartbeat_back_addr":"172.21.15.153:6807/2033986544","heartbeat_front_addr":"172.21.15.153:6805/2033986544","state":["exists","up"]},{"osd":3,"uuid":"0bfd2e0c-53a8-48e4-8604-0ce3cca49fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6811","nonce":931675439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6813","nonce":931675439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6817","nonce":931675439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":931675439},{"type":"v1","addr":"172.21.15.92:6815","nonce":931675439}]},"public_addr":"172.21.15.92:6811/931675439","cluster_addr":"172.21.15.92:6813/931675439","heartbeat_back_addr":"172.21.15.92:6817/931675439","heartbeat_front_addr":"172.21.15.92:6815/931675439","state":["exists","up"]},{"osd":4,"uuid":"b9039a81-37d0-4a40-8050-394638ea88a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6809","nonce":712194146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6811","nonce":712194146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6815","nonce":712194146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":712194146},{"type":"v1","addr":"172.21.15.110:6813","nonce":712194146}]},"public_addr":"172.21.15.110:6809/712194146","cluster_addr":"172.21.15.110:6811/712194146","heartbeat_back_addr":"172.21.15.110:6815/712194146","heartbeat_front_addr":"172.21.15.110:6813/712194146","state":["exists","up"]},{"osd":5,"uuid":"a0472991-0835-438f-9579-8e10646de923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6809","nonce":1354584850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6811","nonce":1354584850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6815","nonce":1354584850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1354584850},{"type":"v1","addr":"172.21.15.153:6813","nonce":1354584850}]},"public_addr":"172.21.15.153:6809/1354584850","cluster_addr":"172.21.15.153:6811/1354584850","heartbeat_back_addr":"172.21.15.153:6815/1354584850","heartbeat_front_addr":"172.21.15.153:6813/1354584850","state":["exists","up"]},{"osd":6,"uuid":"647ec3ad-3258-47f3-9c07-f242cda2ec4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6819","nonce":433608050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6821","nonce":433608050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6825","nonce":433608050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":433608050},{"type":"v1","addr":"172.21.15.92:6823","nonce":433608050}]},"public_addr":"172.21.15.92:6819/433608050","cluster_addr":"172.21.15.92:6821/433608050","heartbeat_back_addr":"172.21.15.92:6825/433608050","heartbeat_front_addr":"172.21.15.92:6823/433608050","state":["exists","up"]},{"osd":7,"uuid":"5810b58c-93d9-4cf6-bdd0-a2c78f61d9cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6817","nonce":1905314527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6819","nonce":1905314527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6823","nonce":1905314527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1905314527},{"type":"v1","addr":"172.21.15.110:6821","nonce":1905314527}]},"public_addr":"172.21.15.110:6817/1905314527","cluster_addr":"172.21.15.110:6819/1905314527","heartbeat_back_addr":"172.21.15.110:6823/1905314527","heartbeat_front_addr":"172.21.15.110:6821/1905314527","state":["exists","up"]},{"osd":8,"uuid":"37cde37b-22c1-4525-9d66-ca50a29c7060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6817","nonce":979097601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6819","nonce":979097601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6823","nonce":979097601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":979097601},{"type":"v1","addr":"172.21.15.153:6821","nonce":979097601}]},"public_addr":"172.21.15.153:6817/979097601","cluster_addr":"172.21.15.153:6819/979097601","heartbeat_back_addr":"172.21.15.153:6823/979097601","heartbeat_front_addr":"172.21.15.153:6821/979097601","state":["exists","up"]},{"osd":9,"uuid":"c38dd0d9-39fb-4c67-9958-18eeb82ef314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6827","nonce":3782209439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6829","nonce":3782209439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6833","nonce":3782209439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3782209439},{"type":"v1","addr":"172.21.15.92:6831","nonce":3782209439}]},"public_addr":"172.21.15.92:6827/3782209439","cluster_addr":"172.21.15.92:6829/3782209439","heartbeat_back_addr":"172.21.15.92:6833/3782209439","heartbeat_front_addr":"172.21.15.92:6831/3782209439","state":["exists","up"]},{"osd":10,"uuid":"abd86e0e-ad63-46dd-961a-54b939ad22b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6825","nonce":3399897751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6827","nonce":3399897751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6831","nonce":3399897751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3399897751},{"type":"v1","addr":"172.21.15.110:6829","nonce":3399897751}]},"public_addr":"172.21.15.110:6825/3399897751","cluster_addr":"172.21.15.110:6827/3399897751","heartbeat_back_addr":"172.21.15.110:6831/3399897751","heartbeat_front_addr":"172.21.15.110:6829/3399897751","state":["exists","up"]},{"osd":11,"uuid":"74de56f5-f0f9-44d6-ac7a-c7edacd28a4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6825","nonce":3726820847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6827","nonce":3726820847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6831","nonce":3726820847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3726820847},{"type":"v1","addr":"172.21.15.153:6829","nonce":3726820847}]},"public_addr":"172.21.15.153:6825/3726820847","cluster_addr":"172.21.15.153:6827/3726820847","heartbeat_back_addr":"172.21.15.153:6831/3726820847","heartbeat_front_addr":"172.21.15.153:6829/3726820847","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:06:35.672815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:00.227681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:25.870169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:07:50.334056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:15.637695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:08:41.464722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:06.199092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:31.232676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:09:56.741803+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:21.420100+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:10:47.509034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:11:12.697108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/2458669467":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/2872938890":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3563316301":"2024-09-12T22:05:15.603416+0000","172.21.15.92:6801/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/3468435378":"2024-09-12T22:04:52.791314+0000","172.21.15.92:0/125981315":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6801/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/1519869556":"2024-09-12T22:04:52.791314+0000","172.21.15.92:6800/733069068":"2024-09-12T22:05:15.603416+0000","172.21.15.92:0/4186575934":"2024-09-12T22:04:52.791314+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:22:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2ee00a2b80 msgr2=0x7f2ee00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2ee00a2b80 0x7f2ee00a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006fd0 tx=0x7f2ef0008040 comp rx=0 tx=0).stop 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 msgr2=0x7f2ef4197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef4197920 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400b4a0 tx=0x7f2ee400b960 comp rx=0 tx=0).stop 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 shutdown_connections 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f2ee00a2b80 0x7f2ee00a5040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ef4106a70 0x7f2ef419bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef4101410 0x7f2ef4197920 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 --2- 172.21.15.92:0/3434128293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4100a70 0x7f2ef41973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 >> 172.21.15.92:0/3434128293 conn(0x7f2ef40fc790 msgr2=0x7f2ef40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 shutdown_connections 2024-09-11T22:22:44.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:44.956+0000 7f2efc1b7640 1 -- 172.21.15.92:0/3434128293 wait complete. 2024-09-11T22:22:45.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:44 smithi110 ceph-mon[34830]: pgmap v608: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:22:45.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:44 smithi153 ceph-mon[35045]: pgmap v608: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:22:45.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T22:22:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:46.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:45 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/3434128293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:46.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:45 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/3434128293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:46.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:45 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/3434128293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 -- 172.21.15.92:0/2390062365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c10c170 msgr2=0x7fb90c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 --2- 172.21.15.92:0/2390062365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c10c170 0x7fb90c10e610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb904009ca0 tx=0x7fb90402a950 comp rx=0 tx=0).stop 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 -- 172.21.15.92:0/2390062365 shutdown_connections 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 --2- 172.21.15.92:0/2390062365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c10c170 0x7fb90c10e610 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 --2- 172.21.15.92:0/2390062365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb90c103660 0x7fb90c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 --2- 172.21.15.92:0/2390062365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c102d40 0x7fb90c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 -- 172.21.15.92:0/2390062365 >> 172.21.15.92:0/2390062365 conn(0x7fb90c0fc7a0 msgr2=0x7fb90c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.390+0000 7fb9122e3640 1 -- 172.21.15.92:0/2390062365 shutdown_connections 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 -- 172.21.15.92:0/2390062365 wait complete. 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 Processor -- start 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 -- start start 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c102d40 0x7fb90c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb90c103660 0x7fb90c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c10c170 0x7fb90c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c111de0 con 0x7fb90c102d40 2024-09-11T22:22:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c111c60 con 0x7fb90c103660 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.391+0000 7fb9122e3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c111f60 con 0x7fb90c10c170 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c102d40 0x7fb90c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c102d40 0x7fb90c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58282/0 (socket says 172.21.15.92:58282) 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 -- 172.21.15.92:0/249480210 learned_addr learned my addr 172.21.15.92:0/249480210 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90b7fe640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb90c103660 0x7fb90c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb910880640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c10c170 0x7fb90c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 -- 172.21.15.92:0/249480210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c10c170 msgr2=0x7fb90c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c10c170 0x7fb90c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 -- 172.21.15.92:0/249480210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb90c103660 msgr2=0x7fb90c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb90c103660 0x7fb90c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 -- 172.21.15.92:0/249480210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c19d270 con 0x7fb90c102d40 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb90bfff640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c102d40 0x7fb90c1982c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c920 tx=0x7fb8fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb9097fa640 1 -- 172.21.15.92:0/249480210 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc007c70 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb9097fa640 1 -- 172.21.15.92:0/249480210 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc007e10 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c19d560 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb9097fa640 1 -- 172.21.15.92:0/249480210 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc018820 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.392+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c1a4e40 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.393+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c103120 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.395+0000 7fb9097fa640 1 -- 172.21.15.92:0/249480210 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb8fc0189c0 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.395+0000 7fb9097fa640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8f40a2b80 0x7fb8f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.396+0000 7fb90b7fe640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8f40a2b80 0x7fb8f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.396+0000 7fb9097fa640 1 -- 172.21.15.92:0/249480210 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8fc0cf850 con 0x7fb90c102d40 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.396+0000 7fb90b7fe640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8f40a2b80 0x7fb8f40a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0fed30 tx=0x7fb90000b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.399+0000 7fb9097fa640 1 -- 172.21.15.92:0/249480210 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8fc014030 con 0x7fb90c102d40 2024-09-11T22:22:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.600+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb90c199620 con 0x7fb90c102d40 2024-09-11T22:22:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.602+0000 7fb9097fa640 1 -- 172.21.15.92:0/249480210 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7fb8fc09c450 con 0x7fb90c102d40 2024-09-11T22:22:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T22:22:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8f40a2b80 msgr2=0x7fb8f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8f40a2b80 0x7fb8f40a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0fed30 tx=0x7fb90000b2e0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c102d40 msgr2=0x7fb90c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c102d40 0x7fb90c1982c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c920 tx=0x7fb8fc00cde0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 shutdown_connections 2024-09-11T22:22:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fb8f40a2b80 0x7fb8f40a5040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c10c170 0x7fb90c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb90c103660 0x7fb90c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 --2- 172.21.15.92:0/249480210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb90c102d40 0x7fb90c1982c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.607+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 >> 172.21.15.92:0/249480210 conn(0x7fb90c0fc7a0 msgr2=0x7fb90c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.608+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 shutdown_connections 2024-09-11T22:22:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:46.608+0000 7fb9122e3640 1 -- 172.21.15.92:0/249480210 wait complete. 2024-09-11T22:22:46.924 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a[31169]: 2024-09-11T22:22:46.601+0000 7fb197c27640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:22:46.924 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a[72945]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.924 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a[72945]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.924 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mds[72949]: -- 172.21.15.92:0/3414895037 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581cec1bc20 con 0x5581cebf9000 2024-09-11T22:22:46.924 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a[72945]: starting mds.a at 2024-09-11T22:22:46.925 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g[72562]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.925 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g[72562]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.925 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mds[72578]: -- 172.21.15.92:0/2712892536 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56270d3f9860 con 0x56270d3d6800 2024-09-11T22:22:46.925 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g[72562]: starting mds.g at 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c[65369]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c[65369]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mds[65373]: -- 172.21.15.153:0/1835356964 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645ea24d860 con 0x5645ea22a800 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c[65369]: starting mds.c at 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i[65674]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i[65674]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mds[65678]: -- 172.21.15.153:0/1195791216 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a54d5b860 con 0x562a54d38800 2024-09-11T22:22:46.930 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i[65674]: starting mds.i at 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l[64762]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l[64762]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l[64762]: starting mds.l at 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f[65067]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f[65067]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mds[65072]: -- 172.21.15.153:0/3467350632 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559661053860 con 0x559661031400 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f[65067]: starting mds.f at 2024-09-11T22:22:46.931 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mds[65150]: -- 172.21.15.110:0/99449794 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf210c3860 con 0x55bf210a0800 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: starting mds.b at 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e[64539]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e[64539]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mds[64543]: -- 172.21.15.110:0/127950900 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56110e5d9860 con 0x56110e5b6800 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e[64539]: starting mds.e at 2024-09-11T22:22:46.932 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h[64844]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.933 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h[64844]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.933 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mds[64848]: -- 172.21.15.110:0/315869715 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4a0757860 con 0x55f4a0735000 2024-09-11T22:22:46.933 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h[64844]: starting mds.h at 2024-09-11T22:22:46.933 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k[64227]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:46.933 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k[64227]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:46.933 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mds[64231]: -- 172.21.15.110:0/611522749 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56262e96fc20 con 0x56262e94d400 2024-09-11T22:22:46.933 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k[64227]: starting mds.k at 2024-09-11T22:22:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mon[34830]: pgmap v609: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:22:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/249480210' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:22:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mon[34830]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:22:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mon[34830]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:22:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/249480210' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:22:47.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mon[34830]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:22:47.218 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:46 smithi110 ceph-mon[34830]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:22:47.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mon[31173]: pgmap v609: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:22:47.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/249480210' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:22:47.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mon[31173]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:22:47.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mon[31173]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:22:47.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/249480210' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:22:47.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mon[31173]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:22:47.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:46 smithi092 ceph-mon[31173]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:22:47.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mon[35045]: pgmap v609: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:22:47.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/249480210' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:22:47.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mon[35045]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:22:47.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mon[35045]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:22:47.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/249480210' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:22:47.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mon[35045]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:22:47.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:46 smithi153 ceph-mon[35045]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:22:47.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T22:22:47.538 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 -- 172.21.15.92:0/1850036431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc106a80 msgr2=0x7f26dc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 --2- 172.21.15.92:0/1850036431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc106a80 0x7f26dc106e60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009930 tx=0x7f26d802a420 comp rx=0 tx=0).stop 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 -- 172.21.15.92:0/1850036431 shutdown_connections 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 --2- 172.21.15.92:0/1850036431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26dc101420 0x7f26dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 --2- 172.21.15.92:0/1850036431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc100a80 0x7f26dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 --2- 172.21.15.92:0/1850036431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc106a80 0x7f26dc106e60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 -- 172.21.15.92:0/1850036431 >> 172.21.15.92:0/1850036431 conn(0x7f26dc0fc7a0 msgr2=0x7f26dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 -- 172.21.15.92:0/1850036431 shutdown_connections 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.000+0000 7f26e3f34640 1 -- 172.21.15.92:0/1850036431 wait complete. 2024-09-11T22:22:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 Processor -- start 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 -- start start 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc100a80 0x7f26dc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc101420 0x7f26dc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26dc106a80 0x7f26dc1a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc110d30 con 0x7f26dc100a80 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc110bb0 con 0x7f26dc106a80 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e3f34640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc110eb0 con 0x7f26dc101420 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc100a80 0x7f26dc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc100a80 0x7f26dc199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58344/0 (socket says 172.21.15.92:58344) 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 -- 172.21.15.92:0/2018269237 learned_addr learned my addr 172.21.15.92:0/2018269237 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e24d1640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26dc106a80 0x7f26dc1a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e14cf640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc101420 0x7f26dc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 -- 172.21.15.92:0/2018269237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc101420 msgr2=0x7f26dc1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc101420 0x7f26dc1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 -- 172.21.15.92:0/2018269237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26dc106a80 msgr2=0x7f26dc1a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26dc106a80 0x7f26dc1a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.001+0000 7f26e1cd0640 1 -- 172.21.15.92:0/2018269237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc1a4820 con 0x7f26dc100a80 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.002+0000 7f26e1cd0640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc100a80 0x7f26dc199260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26d8004770 tx=0x7f26d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.002+0000 7f26d2ffd640 1 -- 172.21.15.92:0/2018269237 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d803f070 con 0x7f26dc100a80 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.002+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc1a49f0 con 0x7f26dc100a80 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.002+0000 7f26d2ffd640 1 -- 172.21.15.92:0/2018269237 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26d803b460 con 0x7f26dc100a80 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.002+0000 7f26d2ffd640 1 -- 172.21.15.92:0/2018269237 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8035070 con 0x7f26dc100a80 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.002+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc1a4ea0 con 0x7f26dc100a80 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.003+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26dc100ee0 con 0x7f26dc100a80 2024-09-11T22:22:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.004+0000 7f26d2ffd640 1 -- 172.21.15.92:0/2018269237 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26d803b610 con 0x7f26dc100a80 2024-09-11T22:22:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.005+0000 7f26d2ffd640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26c80a2b80 0x7f26c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.005+0000 7f26e14cf640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26c80a2b80 0x7f26c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.005+0000 7f26d2ffd640 1 -- 172.21.15.92:0/2018269237 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9580+0+0 (secure 0 0 0) 0x7f26d80ef9f0 con 0x7f26dc100a80 2024-09-11T22:22:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.006+0000 7f26e14cf640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26c80a2b80 0x7f26c80a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.010+0000 7f26d2ffd640 1 -- 172.21.15.92:0/2018269237 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26d80bc410 con 0x7f26dc100a80 2024-09-11T22:22:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.209+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f26dc108ff0 con 0x7f26dc100a80 2024-09-11T22:22:48.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.611+0000 7f26d2ffd640 1 -- 172.21.15.92:0/2018269237 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f26d80bbd10 con 0x7f26dc100a80 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26c80a2b80 msgr2=0x7f26c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26c80a2b80 0x7f26c80a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005850 comp rx=0 tx=0).stop 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc100a80 msgr2=0x7f26dc199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc100a80 0x7f26dc199260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26d8004770 tx=0x7f26d80042f0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 shutdown_connections 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f26c80a2b80 0x7f26c80a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26dc106a80 0x7f26dc1a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc101420 0x7f26dc1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 --2- 172.21.15.92:0/2018269237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26dc100a80 0x7f26dc199260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 >> 172.21.15.92:0/2018269237 conn(0x7f26dc0fc7a0 msgr2=0x7f26dc0fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 shutdown_connections 2024-09-11T22:22:48.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:48.614+0000 7f26e3f34640 1 -- 172.21.15.92:0/2018269237 wait complete. 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.92:6838/1542782268,v1:172.21.15.92:6839/1542782268] up:boot 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.92:6840/1026421623,v1:172.21.15.92:6841/1026421623] up:boot 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6832/449388718,v1:172.21.15.110:6833/449388718] up:boot 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6834/1845919129,v1:172.21.15.110:6835/1845919129] up:boot 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6832/1272354150,v1:172.21.15.153:6833/1272354150] up:boot 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6834/2933091009,v1:172.21.15.153:6835/2933091009] up:boot 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6836/3094888939,v1:172.21.15.110:6838/3094888939] up:boot 2024-09-11T22:22:48.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.110:6837/1109701759,v1:172.21.15.110:6839/1109701759] up:boot 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6836/1386407736,v1:172.21.15.153:6837/1386407736] up:boot 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: mds.? [v2:172.21.15.153:6838/1047150294,v1:172.21.15.153:6839/1047150294] up:boot 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:22:48.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:22:48.953 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: pgmap v611: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-11T22:22:48.953 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:48 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2018269237' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:22:48.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.92:6838/1542782268,v1:172.21.15.92:6839/1542782268] up:boot 2024-09-11T22:22:48.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.92:6840/1026421623,v1:172.21.15.92:6841/1026421623] up:boot 2024-09-11T22:22:48.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6832/449388718,v1:172.21.15.110:6833/449388718] up:boot 2024-09-11T22:22:48.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6834/1845919129,v1:172.21.15.110:6835/1845919129] up:boot 2024-09-11T22:22:48.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6832/1272354150,v1:172.21.15.153:6833/1272354150] up:boot 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6834/2933091009,v1:172.21.15.153:6835/2933091009] up:boot 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6836/3094888939,v1:172.21.15.110:6838/3094888939] up:boot 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.110:6837/1109701759,v1:172.21.15.110:6839/1109701759] up:boot 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6836/1386407736,v1:172.21.15.153:6837/1386407736] up:boot 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: mds.? [v2:172.21.15.153:6838/1047150294,v1:172.21.15.153:6839/1047150294] up:boot 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:22:48.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:22:48.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:22:48.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:22:48.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:22:48.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:22:48.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:22:48.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: pgmap v611: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-11T22:22:48.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:48 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2018269237' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:22:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.92:6838/1542782268,v1:172.21.15.92:6839/1542782268] up:boot 2024-09-11T22:22:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.92:6840/1026421623,v1:172.21.15.92:6841/1026421623] up:boot 2024-09-11T22:22:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6832/449388718,v1:172.21.15.110:6833/449388718] up:boot 2024-09-11T22:22:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6834/1845919129,v1:172.21.15.110:6835/1845919129] up:boot 2024-09-11T22:22:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6832/1272354150,v1:172.21.15.153:6833/1272354150] up:boot 2024-09-11T22:22:49.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6834/2933091009,v1:172.21.15.153:6835/2933091009] up:boot 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6836/3094888939,v1:172.21.15.110:6838/3094888939] up:boot 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.110:6837/1109701759,v1:172.21.15.110:6839/1109701759] up:boot 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6836/1386407736,v1:172.21.15.153:6837/1386407736] up:boot 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: mds.? [v2:172.21.15.153:6838/1047150294,v1:172.21.15.153:6839/1047150294] up:boot 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:22:49.040 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:22:49.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:22:49.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='mgr.14150 172.21.15.92:0/925785703' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:22:49.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: pgmap v611: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-11T22:22:49.041 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:48 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2018269237' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:22:49.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T22:22:49.540 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:49.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:49 smithi092 ceph-mon[31173]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:22:49.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:49 smithi092 ceph-mon[31173]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:22:49.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:49 smithi092 ceph-mon[31173]: Cluster is now healthy 2024-09-11T22:22:49.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:49 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/2018269237' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:22:49.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:49 smithi092 ceph-mon[31173]: fsmap 12 up:standby 2024-09-11T22:22:49.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:49 smithi110 ceph-mon[34830]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:22:49.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:49 smithi110 ceph-mon[34830]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:22:49.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:49 smithi110 ceph-mon[34830]: Cluster is now healthy 2024-09-11T22:22:49.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:49 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/2018269237' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:22:49.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:49 smithi110 ceph-mon[34830]: fsmap 12 up:standby 2024-09-11T22:22:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:49 smithi153 ceph-mon[35045]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:22:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:49 smithi153 ceph-mon[35045]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:22:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:49 smithi153 ceph-mon[35045]: Cluster is now healthy 2024-09-11T22:22:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:49 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/2018269237' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:22:50.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:49 smithi153 ceph-mon[35045]: fsmap 12 up:standby 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 -- 172.21.15.92:0/3349037263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 msgr2=0x7f53641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 --2- 172.21.15.92:0/3349037263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f53641029b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f534c009900 tx=0x7f534c02a480 comp rx=0 tx=0).stop 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 -- 172.21.15.92:0/3349037263 shutdown_connections 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 --2- 172.21.15.92:0/3349037263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5364102ef0 0x7f536410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 --2- 172.21.15.92:0/3349037263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f53641029b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 --2- 172.21.15.92:0/3349037263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5364100e20 0x7f5364101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 -- 172.21.15.92:0/3349037263 >> 172.21.15.92:0/3349037263 conn(0x7f53640fc7d0 msgr2=0x7f53640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 -- 172.21.15.92:0/3349037263 shutdown_connections 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.022+0000 7f5368a3c640 1 -- 172.21.15.92:0/3349037263 wait complete. 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 Processor -- start 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 -- start start 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5364100e20 0x7f5364197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f5364197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5364102ef0 0x7f536419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5364110e30 con 0x7f5364102570 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5364110cb0 con 0x7f5364100e20 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5368a3c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5364110fb0 con 0x7f5364102ef0 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f5364197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f5364197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58368/0 (socket says 172.21.15.92:58368) 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 -- 172.21.15.92:0/4000869360 learned_addr learned my addr 172.21.15.92:0/4000869360 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:50.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f536259c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5364100e20 0x7f5364197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5362d9d640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5364102ef0 0x7f536419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 -- 172.21.15.92:0/4000869360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5364102ef0 msgr2=0x7f536419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5364102ef0 0x7f536419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 -- 172.21.15.92:0/4000869360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5364100e20 msgr2=0x7f5364197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5364100e20 0x7f5364197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f535bfff640 1 -- 172.21.15.92:0/4000869360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536419c2c0 con 0x7f5364102570 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.023+0000 7f5362d9d640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5364102ef0 0x7f536419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.024+0000 7f535bfff640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f5364197850 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f534c006010 tx=0x7f534c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.024+0000 7f535b7fe640 1 -- 172.21.15.92:0/4000869360 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c03f030 con 0x7f5364102570 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.024+0000 7f535b7fe640 1 -- 172.21.15.92:0/4000869360 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f534c003cb0 con 0x7f5364102570 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.024+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536419c550 con 0x7f5364102570 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.024+0000 7f535b7fe640 1 -- 172.21.15.92:0/4000869360 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c035070 con 0x7f5364102570 2024-09-11T22:22:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.024+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53641a3e90 con 0x7f5364102570 2024-09-11T22:22:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.025+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5364101200 con 0x7f5364102570 2024-09-11T22:22:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.026+0000 7f535b7fe640 1 -- 172.21.15.92:0/4000869360 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f534c042030 con 0x7f5364102570 2024-09-11T22:22:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.026+0000 7f535b7fe640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f53500a2b80 0x7f53500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.027+0000 7f536259c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f53500a2b80 0x7f53500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.027+0000 7f535b7fe640 1 -- 172.21.15.92:0/4000869360 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9580+0+0 (secure 0 0 0) 0x7f534c0ef6e0 con 0x7f5364102570 2024-09-11T22:22:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.027+0000 7f536259c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f53500a2b80 0x7f53500a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5354005d00 tx=0x7f535400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.031+0000 7f535b7fe640 1 -- 172.21.15.92:0/4000869360 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f534c0bc100 con 0x7f5364102570 2024-09-11T22:22:50.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.197+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5364109090 con 0x7f5364102570 2024-09-11T22:22:50.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.618+0000 7f535b7fe640 1 -- 172.21.15.92:0/4000869360 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f534c0bba00 con 0x7f5364102570 2024-09-11T22:22:50.620 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2024-09-11T22:22:50.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f53500a2b80 msgr2=0x7f53500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f53500a2b80 0x7f53500a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5354005d00 tx=0x7f535400b410 comp rx=0 tx=0).stop 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 msgr2=0x7f5364197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f5364197850 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f534c006010 tx=0x7f534c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 shutdown_connections 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f53500a2b80 0x7f53500a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5364102ef0 0x7f536419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5364102570 0x7f5364197850 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.621+0000 7f5368a3c640 1 --2- 172.21.15.92:0/4000869360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5364100e20 0x7f5364197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:50.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.622+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 >> 172.21.15.92:0/4000869360 conn(0x7f53640fc7d0 msgr2=0x7f53640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:50.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.622+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 shutdown_connections 2024-09-11T22:22:50.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:50.623+0000 7f5368a3c640 1 -- 172.21.15.92:0/4000869360 wait complete. 2024-09-11T22:22:50.961 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:50 smithi092 ceph-mon[31173]: pgmap v612: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-09-11T22:22:50.961 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:50 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4000869360' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:50.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:50 smithi110 ceph-mon[34830]: pgmap v612: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-09-11T22:22:50.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:50 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4000869360' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:50 smithi153 ceph-mon[35045]: pgmap v612: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-09-11T22:22:51.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:50 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4000869360' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:51.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T22:22:51.488 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:51 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/4000869360' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:51 smithi092 ceph-mon[31173]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:22:51.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:51 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/4000869360' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:51.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:51 smithi110 ceph-mon[34830]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:22:52.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 -- 172.21.15.92:0/1098614041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 msgr2=0x7f7104106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 --2- 172.21.15.92:0/1098614041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f7104106e60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009930 tx=0x7f70f402a420 comp rx=0 tx=0).stop 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 -- 172.21.15.92:0/1098614041 shutdown_connections 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 --2- 172.21.15.92:0/1098614041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7104101420 0x7f710410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 --2- 172.21.15.92:0/1098614041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7104100a80 0x7f7104100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 --2- 172.21.15.92:0/1098614041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f7104106e60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 -- 172.21.15.92:0/1098614041 >> 172.21.15.92:0/1098614041 conn(0x7f71040fc7a0 msgr2=0x7f71040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 -- 172.21.15.92:0/1098614041 shutdown_connections 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.998+0000 7f7108bed640 1 -- 172.21.15.92:0/1098614041 wait complete. 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 Processor -- start 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 -- start start 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7104100a80 0x7f710419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7104101420 0x7f710419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f710419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7104110d30 con 0x7f7104106a80 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7104110bb0 con 0x7f7104100a80 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7108bed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7104110eb0 con 0x7f7104101420 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:51.999+0000 7f7102d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f710419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f710419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58386/0 (socket says 172.21.15.92:58386) 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 -- 172.21.15.92:0/16080148 learned_addr learned my addr 172.21.15.92:0/16080148 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f710259c640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7104100a80 0x7f710419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f70fbfff640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7104101420 0x7f710419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 -- 172.21.15.92:0/16080148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7104101420 msgr2=0x7f710419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7104101420 0x7f710419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 -- 172.21.15.92:0/16080148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7104100a80 msgr2=0x7f710419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7104100a80 0x7f710419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 -- 172.21.15.92:0/16080148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710419f4e0 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7102d9d640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f710419ee00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00eaa0 tx=0x7f70fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f70fb7fe640 1 -- 172.21.15.92:0/16080148 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00cdb0 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f70fb7fe640 1 -- 172.21.15.92:0/16080148 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70fc005020 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710419f7d0 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f70fb7fe640 1 -- 172.21.15.92:0/16080148 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0116d0 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.000+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a70b0 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.001+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7104100ee0 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.002+0000 7f70fb7fe640 1 -- 172.21.15.92:0/16080148 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f70fc0051d0 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.003+0000 7f70fb7fe640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f00a2b80 0x7f70f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.003+0000 7f710259c640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f00a2b80 0x7f70f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.003+0000 7f70fb7fe640 1 -- 172.21.15.92:0/16080148 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8997+0+0 (secure 0 0 0) 0x7f70fc0cf550 con 0x7f7104106a80 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.003+0000 7f710259c640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f00a2b80 0x7f70f00a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000c00 tx=0x7f70f4032ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.007+0000 7f70fb7fe640 1 -- 172.21.15.92:0/16080148 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70fc016030 con 0x7f7104106a80 2024-09-11T22:22:52.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:51 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/4000869360' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:52.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:51 smithi153 ceph-mon[35045]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:22:52.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.174+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f710402d7c0 con 0x7f7104106a80 2024-09-11T22:22:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.625+0000 7f70fb7fe640 1 -- 172.21.15.92:0/16080148 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f70fc09c1b0 con 0x7f7104106a80 2024-09-11T22:22:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2024-09-11T22:22:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f00a2b80 msgr2=0x7f70f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f00a2b80 0x7f70f00a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000c00 tx=0x7f70f4032ba0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 msgr2=0x7f710419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f710419ee00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00eaa0 tx=0x7f70fc00ef60 comp rx=0 tx=0).stop 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 shutdown_connections 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7f70f00a2b80 0x7f70f00a5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7104106a80 0x7f710419ee00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7104101420 0x7f710419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 --2- 172.21.15.92:0/16080148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7104100a80 0x7f710419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 >> 172.21.15.92:0/16080148 conn(0x7f71040fc7a0 msgr2=0x7f710406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 shutdown_connections 2024-09-11T22:22:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:52.627+0000 7f7108bed640 1 -- 172.21.15.92:0/16080148 wait complete. 2024-09-11T22:22:52.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:52 smithi110 ceph-mon[34830]: pgmap v614: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:52.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:52 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/16080148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:52.995 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:52 smithi092 ceph-mon[31173]: pgmap v614: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:52.995 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:52 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/16080148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:53.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:52 smithi153 ceph-mon[35045]: pgmap v614: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:53.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:52 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/16080148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:53.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid a6709e28-7089-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-11T22:22:53.524 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:53.943 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:53 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/16080148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:53.943 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:53 smithi092 ceph-mon[31173]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:22:53.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:53 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/16080148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:53.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:53 smithi110 ceph-mon[34830]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:22:54.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:53 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/16080148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:54.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:53 smithi153 ceph-mon[35045]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.023+0000 7fec5fa2e640 1 -- 172.21.15.92:0/2915833093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 msgr2=0x7fec581021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.023+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/2915833093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec581021d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fec48009ca0 tx=0x7fec4802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.023+0000 7fec5fa2e640 1 -- 172.21.15.92:0/2915833093 shutdown_connections 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.023+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/2915833093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58106f20 0x7fec5810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/2915833093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec581027a0 0x7fec581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/2915833093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec581021d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 -- 172.21.15.92:0/2915833093 >> 172.21.15.92:0/2915833093 conn(0x7fec580fd8f0 msgr2=0x7fec580ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 -- 172.21.15.92:0/2915833093 shutdown_connections 2024-09-11T22:22:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 -- 172.21.15.92:0/2915833093 wait complete. 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 Processor -- start 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 -- start start 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec58198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec581027a0 0x7fec58198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106f20 0x7fec5819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec58111df0 con 0x7fec58101df0 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec58111c70 con 0x7fec581027a0 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.024+0000 7fec5fa2e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec58111f70 con 0x7fec58106f20 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec58198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec58198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58406/0 (socket says 172.21.15.92:58406) 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 -- 172.21.15.92:0/840840227 learned_addr learned my addr 172.21.15.92:0/840840227 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5cfc9640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec581027a0 0x7fec58198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5dfcb640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106f20 0x7fec5819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 -- 172.21.15.92:0/840840227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106f20 msgr2=0x7fec5819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106f20 0x7fec5819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 -- 172.21.15.92:0/840840227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec581027a0 msgr2=0x7fec58198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec581027a0 0x7fec58198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 -- 172.21.15.92:0/840840227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5819d300 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5d7ca640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec58198350 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec4802ade0 tx=0x7fec48032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec567fc640 1 -- 172.21.15.92:0/840840227 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48003cf0 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec567fc640 1 -- 172.21.15.92:0/840840227 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec480044f0 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5819d590 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec567fc640 1 -- 172.21.15.92:0/840840227 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48030c40 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.025+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec581a4ed0 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.026+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec581021d0 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.028+0000 7fec567fc640 1 -- 172.21.15.92:0/840840227 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec48004040 con 0x7fec58101df0 2024-09-11T22:22:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.028+0000 7fec567fc640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec400a2b80 0x7fec400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.028+0000 7fec5cfc9640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec400a2b80 0x7fec400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.028+0000 7fec567fc640 1 -- 172.21.15.92:0/840840227 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8496+0+0 (secure 0 0 0) 0x7fec480f08a0 con 0x7fec58101df0 2024-09-11T22:22:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.029+0000 7fec5cfc9640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec400a2b80 0x7fec400a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec580fff20 tx=0x7fec4c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.031+0000 7fec567fc640 1 -- 172.21.15.92:0/840840227 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec480bd700 con 0x7fec58101df0 2024-09-11T22:22:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.198+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec5810a100 con 0x7fec58101df0 2024-09-11T22:22:54.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.631+0000 7fec567fc640 1 -- 172.21.15.92:0/840840227 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7fec480dd0c0 con 0x7fec58101df0 2024-09-11T22:22:54.632 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data_ec' removed 2024-09-11T22:22:54.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec400a2b80 msgr2=0x7fec400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec400a2b80 0x7fec400a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec580fff20 tx=0x7fec4c00b390 comp rx=0 tx=0).stop 2024-09-11T22:22:54.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 msgr2=0x7fec58198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec58198350 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec4802ade0 tx=0x7fec48032ff0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 shutdown_connections 2024-09-11T22:22:54.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:6800/1619835932,v1:172.21.15.92:6801/1619835932] conn(0x7fec400a2b80 0x7fec400a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106f20 0x7fec5819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec581027a0 0x7fec58198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 --2- 172.21.15.92:0/840840227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec58101df0 0x7fec58198350 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.633+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 >> 172.21.15.92:0/840840227 conn(0x7fec580fd8f0 msgr2=0x7fec580fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.634+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 shutdown_connections 2024-09-11T22:22:54.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-11T22:22:54.634+0000 7fec5fa2e640 1 -- 172.21.15.92:0/840840227 wait complete. 2024-09-11T22:22:54.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:54 smithi110 ceph-mon[34830]: pgmap v616: 33 pgs: 33 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:54.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:54 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/840840227' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:54.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:54 smithi092 ceph-mon[31173]: pgmap v616: 33 pgs: 33 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:54.997 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:54 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/840840227' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:54 smithi153 ceph-mon[35045]: pgmap v616: 33 pgs: 33 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:55.039 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:54 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/840840227' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:22:55.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:55.380 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:55.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:55.447 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T22:22:55.447 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:22:55.463 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:22:55.480 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:22:55.505 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T22:22:55.505 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T22:22:55.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a 2024-09-11T22:22:55.633 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:55 smithi092 systemd[1]: Stopping Ceph mon.a for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:22:55.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:55 smithi110 ceph-mon[34830]: from='client.? 172.21.15.92:0/840840227' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:55.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:55 smithi110 ceph-mon[34830]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:22:55.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:55 smithi153 ceph-mon[35045]: from='client.? 172.21.15.92:0/840840227' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:55.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:55 smithi153 ceph-mon[35045]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:22:56.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:55 smithi092 ceph-mon[31173]: from='client.? 172.21.15.92:0/840840227' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:22:56.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:55 smithi092 ceph-mon[31173]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:22:56.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a[31169]: 2024-09-11T22:22:55.818+0000 7fb19d432640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:22:56.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a[31169]: 2024-09-11T22:22:55.818+0000 7fb19d432640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T22:22:56.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:55 smithi092 podman[194874]: 2024-09-11 22:22:55.850024702 +0000 UTC m=+0.145459333 container died 966a88dcdfc4fefe75aba5011c9d75ff907f83b4958aff94f4da125782d31922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T22:22:56.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:56 smithi092 podman[194874]: 2024-09-11 22:22:56.236336449 +0000 UTC m=+0.531771069 container cleanup 966a88dcdfc4fefe75aba5011c9d75ff907f83b4958aff94f4da125782d31922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T22:22:56.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:56 smithi092 bash[194874]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a 2024-09-11T22:22:56.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T22:22:56.655 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:56 smithi092 podman[194886]: 2024-09-11 22:22:56.40484227 +0000 UTC m=+0.550610193 container remove 966a88dcdfc4fefe75aba5011c9d75ff907f83b4958aff94f4da125782d31922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T22:22:56.655 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:56 smithi092 systemd[1]: ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T22:22:56.655 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:56 smithi092 systemd[1]: Stopped Ceph mon.a for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:22:56.655 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 22:22:56 smithi092 systemd[1]: ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.a.service: Consumed 16.323s CPU time. 2024-09-11T22:22:56.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:22:56.679 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T22:22:56.679 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T22:22:56.679 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.b 2024-09-11T22:22:56.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:56 smithi110 systemd[1]: Stopping Ceph mon.b for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:22:57.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-b[34826]: 2024-09-11T22:22:57.012+0000 7f85d012a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:22:57.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-b[34826]: 2024-09-11T22:22:57.012+0000 7f85d012a640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T22:22:57.469 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:57 smithi110 podman[69333]: 2024-09-11 22:22:57.207465802 +0000 UTC m=+0.327880275 container died 8130dd59e95e8ac4e554b4986bc8f2f3e5b54536b913c085d4d86f5ef740d219 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T22:22:57.849 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:57 smithi110 podman[69333]: 2024-09-11 22:22:57.613972847 +0000 UTC m=+0.734387319 container cleanup 8130dd59e95e8ac4e554b4986bc8f2f3e5b54536b913c085d4d86f5ef740d219 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T22:22:57.849 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 22:22:57 smithi110 bash[69333]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-b 2024-09-11T22:22:57.998 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T22:22:58.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:22:58.046 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T22:22:58.046 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T22:22:58.046 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.c 2024-09-11T22:22:58.374 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:58 smithi153 systemd[1]: Stopping Ceph mon.c for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:22:58.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-c[35041]: 2024-09-11T22:22:58.373+0000 7fb631b25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:22:58.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-c[35041]: 2024-09-11T22:22:58.373+0000 7fb631b25640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T22:22:58.789 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:58 smithi153 podman[69618]: 2024-09-11 22:22:58.588939261 +0000 UTC m=+0.336664791 container died 4f85c6c52807125e8d56da53671a66282b382709837b3f1344fcd332dfb3707d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T22:22:59.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:59 smithi153 podman[69618]: 2024-09-11 22:22:59.033351125 +0000 UTC m=+0.781076662 container cleanup 4f85c6c52807125e8d56da53671a66282b382709837b3f1344fcd332dfb3707d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T22:22:59.289 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 11 22:22:59 smithi153 bash[69618]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mon-c 2024-09-11T22:22:59.463 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T22:22:59.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:22:59.517 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T22:22:59.517 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T22:22:59.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.x 2024-09-11T22:22:59.825 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:22:59 smithi092 systemd[1]: Stopping Ceph mgr.x for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:00.134 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:22:59 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x[31397]: 2024-09-11T22:22:59.823+0000 7f8ec5a81640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:00.134 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:22:59 smithi092 podman[195007]: 2024-09-11 22:22:59.855683587 +0000 UTC m=+0.149329894 container died 708a65413e49c0a13d97244ebeabba361a0ca587e687db34d8b69a006c817c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T22:23:00.503 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:23:00 smithi092 podman[195007]: 2024-09-11 22:23:00.242428799 +0000 UTC m=+0.536075105 container cleanup 708a65413e49c0a13d97244ebeabba361a0ca587e687db34d8b69a006c817c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:23:00.503 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 11 22:23:00 smithi092 bash[195007]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-x 2024-09-11T22:23:00.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T22:23:00.698 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:00.698 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T22:23:00.698 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T22:23:00.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.y 2024-09-11T22:23:01.040 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:23:00 smithi110 systemd[1]: Stopping Ceph mgr.y for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:01.303 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:23:01 smithi110 podman[69457]: 2024-09-11 22:23:01.040158964 +0000 UTC m=+0.147955338 container died ab1b1fe2c08bb6529f65fb2d2f3f2f1cba392dea38175190e1a679fb5d897781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T22:23:01.697 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:23:01 smithi110 podman[69457]: 2024-09-11 22:23:01.420028191 +0000 UTC m=+0.527824570 container cleanup ab1b1fe2c08bb6529f65fb2d2f3f2f1cba392dea38175190e1a679fb5d897781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T22:23:01.697 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 22:23:01 smithi110 bash[69457]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-y 2024-09-11T22:23:01.880 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T22:23:01.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:01.917 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T22:23:01.917 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T22:23:01.917 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.z 2024-09-11T22:23:02.260 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:23:01 smithi153 systemd[1]: Stopping Ceph mgr.z for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:02.539 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:23:02 smithi153 podman[69743]: 2024-09-11 22:23:02.259871784 +0000 UTC m=+0.145459192 container died eb4416760230fe3e3571946b8b74ea2b5e66c2f29cb1d7c6d0152e76e6cd0713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T22:23:02.933 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:23:02 smithi153 podman[69743]: 2024-09-11 22:23:02.681505794 +0000 UTC m=+0.567093212 container cleanup eb4416760230fe3e3571946b8b74ea2b5e66c2f29cb1d7c6d0152e76e6cd0713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T22:23:02.934 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 11 22:23:02 smithi153 bash[69743]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mgr-z 2024-09-11T22:23:03.074 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T22:23:03.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:03.111 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T22:23:03.111 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T22:23:03.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.0 2024-09-11T22:23:03.430 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:03 smithi092 systemd[1]: Stopping Ceph osd.0 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:03.768 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-11T22:23:03.429+0000 7fbad5b58640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:03.768 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-11T22:23:03.429+0000 7fbad5b58640 -1 osd.0 98 *** Got signal Terminated *** 2024-09-11T22:23:03.768 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0[45348]: 2024-09-11T22:23:03.429+0000 7fbad5b58640 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:09.018 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:08 smithi092 podman[195130]: 2024-09-11 22:23:08.872680938 +0000 UTC m=+5.568394447 container died 64447f5f06c1731867371a5273c1c12ab9cb4962352935833072d19787928e07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T22:23:09.492 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:09 smithi092 podman[195130]: 2024-09-11 22:23:09.224541592 +0000 UTC m=+5.920255090 container cleanup 64447f5f06c1731867371a5273c1c12ab9cb4962352935833072d19787928e07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T22:23:09.493 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:09 smithi092 bash[195130]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0 2024-09-11T22:23:09.768 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:09 smithi092 podman[195145]: 2024-09-11 22:23:09.492638513 +0000 UTC m=+0.614025905 container remove 64447f5f06c1731867371a5273c1c12ab9cb4962352935833072d19787928e07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T22:23:10.096 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:09 smithi092 podman[195236]: 2024-09-11 22:23:09.712405238 +0000 UTC m=+0.015928880 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:23:10.097 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:09 smithi092 podman[195236]: 2024-09-11 22:23:09.835495765 +0000 UTC m=+0.139019449 container create b400d5dc0ff9b217374a66383a32c17ac82dc7ee6b68f1134ce85994fcfe0858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T22:23:10.097 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:10 smithi092 podman[195236]: 2024-09-11 22:23:10.096523489 +0000 UTC m=+0.400047134 container init b400d5dc0ff9b217374a66383a32c17ac82dc7ee6b68f1134ce85994fcfe0858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T22:23:10.519 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:10 smithi092 podman[195236]: 2024-09-11 22:23:10.100099279 +0000 UTC m=+0.403622929 container start b400d5dc0ff9b217374a66383a32c17ac82dc7ee6b68f1134ce85994fcfe0858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T22:23:10.519 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:10 smithi092 podman[195236]: 2024-09-11 22:23:10.1882671 +0000 UTC m=+0.491790747 container attach b400d5dc0ff9b217374a66383a32c17ac82dc7ee6b68f1134ce85994fcfe0858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T22:23:10.519 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 22:23:10 smithi092 podman[195236]: 2024-09-11 22:23:10.284889251 +0000 UTC m=+0.588412894 container died b400d5dc0ff9b217374a66383a32c17ac82dc7ee6b68f1134ce85994fcfe0858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T22:23:10.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T22:23:10.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:10.969 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T22:23:10.969 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T22:23:10.969 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.1 2024-09-11T22:23:11.275 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:11 smithi110 systemd[1]: Stopping Ceph osd.1 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:11.717 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1[39273]: 2024-09-11T22:23:11.274+0000 7f4f53d7e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:11.717 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1[39273]: 2024-09-11T22:23:11.274+0000 7f4f53d7e640 -1 osd.1 98 *** Got signal Terminated *** 2024-09-11T22:23:11.717 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1[39273]: 2024-09-11T22:23:11.274+0000 7f4f53d7e640 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:17.110 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:16 smithi110 podman[69591]: 2024-09-11 22:23:16.797012497 +0000 UTC m=+5.633445524 container died 2a830a3e651b85699b8323a29bd609ea0977f06e27b756d00d00694efa4d976c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:23:17.443 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:17 smithi110 podman[69591]: 2024-09-11 22:23:17.208558099 +0000 UTC m=+6.044991124 container cleanup 2a830a3e651b85699b8323a29bd609ea0977f06e27b756d00d00694efa4d976c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T22:23:17.444 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:17 smithi110 bash[69591]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1 2024-09-11T22:23:17.717 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:17 smithi110 podman[69606]: 2024-09-11 22:23:17.443562974 +0000 UTC m=+0.647757737 container remove 2a830a3e651b85699b8323a29bd609ea0977f06e27b756d00d00694efa4d976c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T22:23:18.172 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:17 smithi110 podman[69694]: 2024-09-11 22:23:17.781353779 +0000 UTC m=+0.015540398 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:23:18.172 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:17 smithi110 podman[69694]: 2024-09-11 22:23:17.911117638 +0000 UTC m=+0.145304253 container create 27d51ae57922bbed226afe86f5a8ce4e6f0f4f65b436e0a2b53f8b1ff1d8a9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T22:23:18.467 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:18 smithi110 podman[69694]: 2024-09-11 22:23:18.172099852 +0000 UTC m=+0.406286469 container init 27d51ae57922bbed226afe86f5a8ce4e6f0f4f65b436e0a2b53f8b1ff1d8a9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T22:23:18.467 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:18 smithi110 podman[69694]: 2024-09-11 22:23:18.175582737 +0000 UTC m=+0.409769358 container start 27d51ae57922bbed226afe86f5a8ce4e6f0f4f65b436e0a2b53f8b1ff1d8a9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T22:23:18.467 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:18 smithi110 podman[69694]: 2024-09-11 22:23:18.263909613 +0000 UTC m=+0.498096228 container attach 27d51ae57922bbed226afe86f5a8ce4e6f0f4f65b436e0a2b53f8b1ff1d8a9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:23:18.468 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 22:23:18 smithi110 podman[69694]: 2024-09-11 22:23:18.360161007 +0000 UTC m=+0.594347634 container died 27d51ae57922bbed226afe86f5a8ce4e6f0f4f65b436e0a2b53f8b1ff1d8a9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T22:23:19.058 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T22:23:19.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:19.098 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T22:23:19.098 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T22:23:19.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.2 2024-09-11T22:23:19.420 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:19 smithi153 systemd[1]: Stopping Ceph osd.2 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:19.789 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2[39997]: 2024-09-11T22:23:19.417+0000 7fa6a210f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:19.789 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2[39997]: 2024-09-11T22:23:19.417+0000 7fa6a210f640 -1 osd.2 98 *** Got signal Terminated *** 2024-09-11T22:23:19.789 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2[39997]: 2024-09-11T22:23:19.417+0000 7fa6a210f640 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:25.137 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:24 smithi153 podman[69881]: 2024-09-11 22:23:24.826545603 +0000 UTC m=+5.534894507 container died bd9ab63edc215c6d4a147129e58323dbb4e41e533b306dcf75bb781ec5bffa49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T22:23:25.479 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:25 smithi153 podman[69881]: 2024-09-11 22:23:25.2191848 +0000 UTC m=+5.927533704 container cleanup bd9ab63edc215c6d4a147129e58323dbb4e41e533b306dcf75bb781ec5bffa49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T22:23:25.479 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:25 smithi153 bash[69881]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2 2024-09-11T22:23:25.787 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:25 smithi153 podman[69897]: 2024-09-11 22:23:25.479104297 +0000 UTC m=+0.646161014 container remove bd9ab63edc215c6d4a147129e58323dbb4e41e533b306dcf75bb781ec5bffa49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T22:23:26.039 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:25 smithi153 podman[69984]: 2024-09-11 22:23:25.690483408 +0000 UTC m=+0.016028758 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:23:26.039 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:25 smithi153 podman[69984]: 2024-09-11 22:23:25.821863087 +0000 UTC m=+0.147408438 container create 70bf2387e9a2fefa2a5265621d15ea0a66faef405d8896ff1df033f0d2aa3d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T22:23:26.539 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:26 smithi153 podman[69984]: 2024-09-11 22:23:26.07524195 +0000 UTC m=+0.400787308 container init 70bf2387e9a2fefa2a5265621d15ea0a66faef405d8896ff1df033f0d2aa3d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T22:23:26.539 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:26 smithi153 podman[69984]: 2024-09-11 22:23:26.078637844 +0000 UTC m=+0.404183198 container start 70bf2387e9a2fefa2a5265621d15ea0a66faef405d8896ff1df033f0d2aa3d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T22:23:26.539 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:26 smithi153 podman[69984]: 2024-09-11 22:23:26.174800981 +0000 UTC m=+0.500346343 container attach 70bf2387e9a2fefa2a5265621d15ea0a66faef405d8896ff1df033f0d2aa3d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T22:23:26.539 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 11 22:23:26 smithi153 podman[69984]: 2024-09-11 22:23:26.261961161 +0000 UTC m=+0.587506517 container died 70bf2387e9a2fefa2a5265621d15ea0a66faef405d8896ff1df033f0d2aa3d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T22:23:27.085 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T22:23:27.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:27.129 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T22:23:27.129 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T22:23:27.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.3 2024-09-11T22:23:27.444 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:27 smithi092 systemd[1]: Stopping Ceph osd.3 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:27.701 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:27 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:23:27.443+0000 7fb1c6b42640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:27.701 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:27 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:23:27.443+0000 7fb1c6b42640 -1 osd.3 98 *** Got signal Terminated *** 2024-09-11T22:23:27.701 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:27 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:23:27.443+0000 7fb1c6b42640 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:30.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:29 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:29.771+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:31.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:30 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:30.752+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:31.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:31 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:31.137+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:31.518 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:31 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:23:31.113+0000 7fb1bf150640 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:04.858775+0000 front 2024-09-11T22:23:04.858750+0000 (oldest deadline 2024-09-11T22:23:30.158862+0000) 2024-09-11T22:23:32.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:31 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:31.751+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:32.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:32 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:32.106+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:32.519 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:32 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3[52106]: 2024-09-11T22:23:32.137+0000 7fb1bf150640 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:04.858775+0000 front 2024-09-11T22:23:04.858750+0000 (oldest deadline 2024-09-11T22:23:30.158862+0000) 2024-09-11T22:23:33.018 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:32 smithi092 podman[195339]: 2024-09-11 22:23:32.794319448 +0000 UTC m=+5.470042586 container died 2f5398a1783364ee6de194ac04342f35bf651722321c457337a489602d13c6c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:23:33.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:32 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:32.742+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:33.395 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:33 smithi092 podman[195339]: 2024-09-11 22:23:33.168981236 +0000 UTC m=+5.844704401 container cleanup 2f5398a1783364ee6de194ac04342f35bf651722321c457337a489602d13c6c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-11T22:23:33.395 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:33 smithi092 bash[195339]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3 2024-09-11T22:23:33.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:33 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:33.089+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:33.729 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:33 smithi092 podman[195366]: 2024-09-11 22:23:33.395196723 +0000 UTC m=+0.596465209 container remove 2f5398a1783364ee6de194ac04342f35bf651722321c457337a489602d13c6c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T22:23:33.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:33.302+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:33.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:33.529+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:33.990 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:33 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:33.766+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:33.990 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:33 smithi092 podman[195456]: 2024-09-11 22:23:33.631952096 +0000 UTC m=+0.015755809 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:23:33.990 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:33 smithi092 podman[195456]: 2024-09-11 22:23:33.737722874 +0000 UTC m=+0.121526577 container create 8aba0345cae2573fe9a1ee53ee9d5a8c65d2ccc563d515fdc48bd80328d5dde5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T22:23:33.991 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:33 smithi092 podman[195456]: 2024-09-11 22:23:33.990510693 +0000 UTC m=+0.374314404 container init 8aba0345cae2573fe9a1ee53ee9d5a8c65d2ccc563d515fdc48bd80328d5dde5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:23:34.269 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:33 smithi092 podman[195456]: 2024-09-11 22:23:33.993971915 +0000 UTC m=+0.377775617 container start 8aba0345cae2573fe9a1ee53ee9d5a8c65d2ccc563d515fdc48bd80328d5dde5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T22:23:34.269 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:34 smithi092 podman[195456]: 2024-09-11 22:23:34.090583776 +0000 UTC m=+0.474387484 container attach 8aba0345cae2573fe9a1ee53ee9d5a8c65d2ccc563d515fdc48bd80328d5dde5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T22:23:34.269 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 22:23:34 smithi092 podman[195456]: 2024-09-11 22:23:34.178831234 +0000 UTC m=+0.562634952 container died 8aba0345cae2573fe9a1ee53ee9d5a8c65d2ccc563d515fdc48bd80328d5dde5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T22:23:34.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:34 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:34.130+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:34.513 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:34.263+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:34.514 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:34.156+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:34.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:34.513+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:34.819 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:34 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:34.773+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:34.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T22:23:34.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:34.888 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T22:23:34.888 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T22:23:34.888 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.4 2024-09-11T22:23:35.197 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:35 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:35.164+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:35.197 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:34 smithi110 systemd[1]: Stopping Ceph osd.4 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:35.467 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:35 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:23:35.196+0000 7fb66b99c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:35.467 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:35 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:23:35.196+0000 7fb66b99c640 -1 osd.4 98 *** Got signal Terminated *** 2024-09-11T22:23:35.467 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:35 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:23:35.196+0000 7fb66b99c640 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:35.493 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:35 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:35.256+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:35.493 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:35 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:35.119+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:35.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:35 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:35.492+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:36.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:35 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:35.791+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:36.459 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:36 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:36.252+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:36.460 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:36 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:36.083+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:36.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:36 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:36.187+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:36.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:36 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:36.459+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:37.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:36 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:36.797+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:37.415 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:37 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:37.273+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:37.416 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:37 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:37.122+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:37.467 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:37 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:23:37.399+0000 7fb667fb2640 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.197025+0000 front 2024-09-11T22:23:12.197062+0000 (oldest deadline 2024-09-11T22:23:36.897260+0000) 2024-09-11T22:23:37.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:37 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:37.164+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:37.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:37 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:37.415+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:37.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:37 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:37.415+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:38.172 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:37 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:37.771+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:38.430 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:38 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:38.146+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:38.431 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:38 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:38.294+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:38.433 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:38 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:38.161+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:38.717 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:38 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:23:38.432+0000 7fb667fb2640 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.197025+0000 front 2024-09-11T22:23:12.197062+0000 (oldest deadline 2024-09-11T22:23:36.897260+0000) 2024-09-11T22:23:38.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:38 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:38.429+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:38.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:38 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:38.429+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:39.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:38 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:38.723+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:39.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:38 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:38.723+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:39.437 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:39 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:39.170+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:39.437 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:39 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:39.181+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:39.437 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:39 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:39.181+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:39.441 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:39 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:39.153+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:39.441 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:39 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:39.153+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:39.441 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:39 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:39.303+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:39.441 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:39 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:39.303+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:39.717 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:39 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4[45865]: 2024-09-11T22:23:39.435+0000 7fb667fb2640 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.197025+0000 front 2024-09-11T22:23:12.197062+0000 (oldest deadline 2024-09-11T22:23:36.897260+0000) 2024-09-11T22:23:39.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:39 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:39.440+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:39.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:39 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:39.440+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:40.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:39 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:39.714+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:40.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:39 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:39.714+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:40.019 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:39 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:39.623+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:40.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:40 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:40.142+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:40.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:40 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:40.169+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:40.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:40 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:40.169+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:40.539 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:40 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:40.443+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:40.539 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:40 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:40.443+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:40.539 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:40 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:40.200+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:40.539 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:40 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:40.200+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:40.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:40 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:40.274+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:40.540 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:40 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:40.274+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:40.967 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:40 smithi110 podman[69823]: 2024-09-11 22:23:40.671215574 +0000 UTC m=+5.590078453 container died a3fadcc2760d46be4a2fa3cbca10782db93bc9f5a53acd10cc3c19a190fce8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:23:41.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:40 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:40.662+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:41.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:40 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:40.738+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:41.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:40 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:40.738+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:41.332 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:41 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:41.098+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:41.332 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:41 smithi110 podman[69823]: 2024-09-11 22:23:41.072404194 +0000 UTC m=+5.991267065 container cleanup a3fadcc2760d46be4a2fa3cbca10782db93bc9f5a53acd10cc3c19a190fce8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:23:41.332 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:41 smithi110 bash[69823]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4 2024-09-11T22:23:41.332 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:41 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:41.128+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:41.333 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:41 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:41.128+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:41.468 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:41 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:41.191+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:41.468 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:41 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:41.191+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:41.468 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:41 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:41.290+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:41.469 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:41 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:41.290+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:41.682 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:41 smithi110 podman[69842]: 2024-09-11 22:23:41.332148504 +0000 UTC m=+0.662817503 container remove a3fadcc2760d46be4a2fa3cbca10782db93bc9f5a53acd10cc3c19a190fce8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T22:23:41.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:41 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:41.467+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:41.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:41 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:41.467+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:41.935 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:41 smithi110 podman[69933]: 2024-09-11 22:23:41.58481413 +0000 UTC m=+0.016181292 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:23:41.936 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:41 smithi110 podman[69933]: 2024-09-11 22:23:41.699511707 +0000 UTC m=+0.130878855 container create a1691585d4e5e06922361d963ddc4bd7060c1590eb69d2021a509f58979c43bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:23:41.936 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:41 smithi110 podman[69933]: 2024-09-11 22:23:41.935410319 +0000 UTC m=+0.366777467 container init a1691585d4e5e06922361d963ddc4bd7060c1590eb69d2021a509f58979c43bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:23:42.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:41.751+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:42.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:41.751+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:42.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:41 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:41.651+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:42.191 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:42 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:42.114+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:42.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:42 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:42.097+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:42.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:42 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:42.097+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:42.192 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:41 smithi110 podman[69933]: 2024-09-11 22:23:41.938964133 +0000 UTC m=+0.370331284 container start a1691585d4e5e06922361d963ddc4bd7060c1590eb69d2021a509f58979c43bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:23:42.192 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:42 smithi110 podman[69933]: 2024-09-11 22:23:42.035276465 +0000 UTC m=+0.466643622 container attach a1691585d4e5e06922361d963ddc4bd7060c1590eb69d2021a509f58979c43bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T22:23:42.192 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 22:23:42 smithi110 podman[69933]: 2024-09-11 22:23:42.120975501 +0000 UTC m=+0.552342660 container died a1691585d4e5e06922361d963ddc4bd7060c1590eb69d2021a509f58979c43bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:23:42.484 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:42.154+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:42.484 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:42.154+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:42.485 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:42.276+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:42.485 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:42.276+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:42.767 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T22:23:42.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:42.484+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:42.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:42 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:42.484+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:42.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:42.805 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T22:23:42.805 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T22:23:42.805 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.5 2024-09-11T22:23:43.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:42 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:42.626+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:43.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:42 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:42.728+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:43.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:42 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:42.728+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:43.112 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:42 smithi153 systemd[1]: Stopping Ceph osd.5 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:43.435 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:43.277+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:43.436 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:43.277+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:43.436 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:43.111+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:43.436 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:43.111+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:43.436 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:43.123+0000 7f043dd95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:43.436 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:43.123+0000 7f043dd95640 -1 osd.5 98 *** Got signal Terminated *** 2024-09-11T22:23:43.436 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:43.123+0000 7f043dd95640 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:43.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:43 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:43.113+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:43.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:43 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:43.147+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:43.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:43 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:43.147+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:43.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:43.435+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:43.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:43 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:43.435+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:44.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:43 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:43.621+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:44.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:43 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:43.731+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:44.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:43 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:43.731+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:44.453 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:44 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:44.267+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:44.454 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:44 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:44.267+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:44.454 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:44 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:44.125+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:44.454 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:44 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:44.125+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:44.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:44 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:44.144+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:44.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:44 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:44.155+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:44.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:44 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:44.155+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:44.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:44 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:44.453+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:44.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:44 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:44.453+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:45.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:44 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:44.651+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:45.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:44 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:44.745+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:45.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:44 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:44.746+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:45.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:45 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:45.141+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:45.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:45 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:45.146+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:45.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:45 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:45.146+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:45.477 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:45 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:45.276+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:45.477 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:45 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:45.276+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:45.477 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:45 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:45.079+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:45.477 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:45 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:45.079+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:45.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:45 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:45.477+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:45.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:45 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:45.477+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:45.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:45 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:45.477+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:46.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:45 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:45.686+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:46.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:45 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:45.740+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:46.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:45 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:45.740+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:46.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:46.135+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:46.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:46.185+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:46.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:46 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:46.185+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:46.496 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:46.271+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:46.496 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:46.271+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:46.496 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:46.120+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:46.496 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:46.120+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:46.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:46.495+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:46.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:46.495+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:46.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:46 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:46.495+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:47.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:46.704+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:47.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:46.739+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:47.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:46 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:46.739+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:47.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:47 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:47.145+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:47.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:47 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:47.152+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:47.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:47 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:47.152+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:47.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:47 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:47.152+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:47.524 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:47 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:47.287+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:47.524 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:47 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:47.287+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:47.524 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:47 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:47.098+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:47.524 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:47 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:47.098+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:47.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:47 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:47.523+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:47.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:47 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:47.523+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:47.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:47 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:47.523+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:48.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:47 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:47.677+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:48.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:47 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:47.780+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:48.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:47 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:47.780+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:48.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:48 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:48.146+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:48.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:48 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:48.146+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:48.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:48 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:48.128+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:48.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:48 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:48.128+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:48.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:48 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:48.128+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:48.539 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:48 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:48.064+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:08.209036+0000 front 2024-09-11T22:23:08.208943+0000 (oldest deadline 2024-09-11T22:23:34.009626+0000) 2024-09-11T22:23:48.539 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:48 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5[46839]: 2024-09-11T22:23:48.064+0000 7f04363a3640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.010134+0000 front 2024-09-11T22:23:14.010076+0000 (oldest deadline 2024-09-11T22:23:38.710173+0000) 2024-09-11T22:23:48.540 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:48 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:48.248+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:48.540 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:48 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:48.248+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:48.974 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:48 smithi153 podman[70099]: 2024-09-11 22:23:48.669513489 +0000 UTC m=+5.671461190 container died 22c2a138371f03f51aeeec05bda1cede000c683beee82edcc62d4cf0703523f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T22:23:48.974 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:48 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:48.541+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:48.974 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:48 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:48.541+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:48.974 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:48 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:48.541+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:49.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:48.639+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:49.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:48.639+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:49.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:48.809+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:49.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:48.809+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:49.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:48.809+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:49.257 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:49 smithi153 podman[70099]: 2024-09-11 22:23:49.065200495 +0000 UTC m=+6.067148202 container cleanup 22c2a138371f03f51aeeec05bda1cede000c683beee82edcc62d4cf0703523f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:23:49.258 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:49 smithi153 bash[70099]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5 2024-09-11T22:23:49.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:49 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:49.189+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:49.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:49 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:49.189+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:49.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:49 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:49.115+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:49.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:49 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:49.115+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:49.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:49 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:49.115+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:49.539 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:49 smithi153 podman[70115]: 2024-09-11 22:23:49.341926101 +0000 UTC m=+0.671263262 container remove 22c2a138371f03f51aeeec05bda1cede000c683beee82edcc62d4cf0703523f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T22:23:49.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:49 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:49.257+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:49.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:49 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:49.257+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:49.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:49 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:49.257+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:49.838 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:49 smithi153 podman[70201]: 2024-09-11 22:23:49.581917862 +0000 UTC m=+0.015957153 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:23:49.838 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:49 smithi153 podman[70201]: 2024-09-11 22:23:49.701179402 +0000 UTC m=+0.135218693 container create 3a9f2c7ab12ea8bb2b9dcd1cd14ac7c55cbdb11e871af955058cb0fb00b2a104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T22:23:49.838 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:49 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:49.570+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:49.839 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:49 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:49.570+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:49.839 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:49 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:49.570+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:50.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:49.613+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:50.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:49.613+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:50.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:49.817+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:50.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:49.817+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:50.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:49 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:49.817+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:50.073 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:50 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:50.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:50.073 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:50 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:50.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:50.073 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:50 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:50.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:50.279 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:49 smithi153 podman[70201]: 2024-09-11 22:23:49.962200264 +0000 UTC m=+0.396239555 container init 3a9f2c7ab12ea8bb2b9dcd1cd14ac7c55cbdb11e871af955058cb0fb00b2a104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T22:23:50.279 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:49 smithi153 podman[70201]: 2024-09-11 22:23:49.966046389 +0000 UTC m=+0.400085692 container start 3a9f2c7ab12ea8bb2b9dcd1cd14ac7c55cbdb11e871af955058cb0fb00b2a104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T22:23:50.279 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:50 smithi153 podman[70201]: 2024-09-11 22:23:50.045476049 +0000 UTC m=+0.479515353 container attach 3a9f2c7ab12ea8bb2b9dcd1cd14ac7c55cbdb11e871af955058cb0fb00b2a104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:23:50.279 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 11 22:23:50 smithi153 podman[70201]: 2024-09-11 22:23:50.162148522 +0000 UTC m=+0.596187822 container died 3a9f2c7ab12ea8bb2b9dcd1cd14ac7c55cbdb11e871af955058cb0fb00b2a104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:23:50.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:50 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:50.159+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:50.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:50 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:50.159+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:50.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:50 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:50.278+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:50.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:50 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:50.278+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:50.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:50 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:50.278+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:50.785 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T22:23:50.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:50.825 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T22:23:50.825 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T22:23:50.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.6 2024-09-11T22:23:50.852 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:50.581+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:50.852 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:50.581+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:50.852 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:50.804+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:50.852 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:50.804+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:50.852 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:50 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:50.804+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:51.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:50 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:50.619+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:51.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:50 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:50.619+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:51.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:50 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:50.619+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:51.138 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:50 smithi092 systemd[1]: Stopping Ceph osd.6 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:51.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:51 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:51.192+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:51.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:51 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:51.192+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:51.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:51 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:51.084+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:51.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:51 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:51.084+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:51.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:51 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:51.084+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:51.518 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:51.138+0000 7ff80e01f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:51.518 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:51.138+0000 7ff80e01f640 -1 osd.6 98 *** Got signal Terminated *** 2024-09-11T22:23:51.518 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:51.138+0000 7ff80e01f640 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:51.583 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:51 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:51.296+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:51.583 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:51 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:51.296+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:51.583 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:51 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:51.296+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:51.829 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:51.539+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:51.829 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:51.539+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:52.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:51 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:51.582+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:52.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:51 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:51.582+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:52.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:51 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:51.582+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:52.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:51.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:52.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:51.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:52.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:51.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:52.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:52 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:52.222+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:52.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:52 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:52.222+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:52.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:52 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:52.093+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:52.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:52 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:52.093+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:52.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:52 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:52.093+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:52.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:52 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:52.265+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:52.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:52 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:52.265+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:52.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:52 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:52.265+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:53.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:52 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:52.578+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:53.019 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:52 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:52.578+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:53.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:52 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:52.805+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:53.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:52 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:52.805+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:53.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:52 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:52.805+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:53.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:52 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:52.615+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:53.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:52 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:52.615+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:53.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:52 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:52.615+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:53.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:53 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:53.247+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:53.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:53 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:53.247+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:53.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:53 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:53.059+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:53.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:53 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:53.059+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:53.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:53 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:53.059+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:53.625 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:53 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:53.290+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:53.626 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:53 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:53.290+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:53.626 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:53 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:53.290+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:54.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:53 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:53.575+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:54.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:53 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:53.575+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:54.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:53 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:53.823+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:54.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:53 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:53.823+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:54.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:53 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:53.823+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:54.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:53 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:53.625+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:54.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:53 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:53.625+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:54.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:53 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:53.625+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:54.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:54 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:54.281+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:54.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:54 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:54.281+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:54.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:54 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:54.281+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:23:54.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:54 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:54.095+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:54.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:54 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:54.095+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:54.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:54 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:54.095+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:54.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:54 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:54.095+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:23:54.624 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:54 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:54.335+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:54.624 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:54 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:54.335+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:54.625 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:54 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:54.335+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:54.625 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:54 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:54.335+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:23:55.018 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:54.549+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:55.019 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:54.549+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:55.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:54.774+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:55.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:54.774+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:55.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:54 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:54.774+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:55.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:54 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:54.623+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:55.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:54 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:54.623+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:55.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:54 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:54.623+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:55.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:55 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:55.273+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:55.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:55 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:55.273+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:55.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:55 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:55.273+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:23:55.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:55 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:55.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:55.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:55 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:55.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:55.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:55 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:55.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:55.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:55 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:55.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:23:55.619 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:55.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:55.619 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:55.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:55.620 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:55.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:55.620 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:55.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:23:55.815 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:55.521+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:14.640641+0000 front 2024-09-11T22:23:14.640868+0000 (oldest deadline 2024-09-11T22:23:39.340692+0000) 2024-09-11T22:23:55.815 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6[59042]: 2024-09-11T22:23:55.521+0000 7ff80662d640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:23.942552+0000 front 2024-09-11T22:23:23.942329+0000 (oldest deadline 2024-09-11T22:23:48.042187+0000) 2024-09-11T22:23:56.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:55.619+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:56.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:55.619+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:56.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:55 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:55.619+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:56.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:55.815+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:56.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:55.815+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:56.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:55.815+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:56.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:55 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:55.815+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:23:56.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:56.293+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:56.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:56.293+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:56.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:56.293+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:23:56.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:56.097+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:56.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:56.097+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:56.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:56.097+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:56.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:56.097+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:23:56.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:56 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:56.363+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:56.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:56 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:56.363+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:56.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:56 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:56.363+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:56.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:56 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:56.363+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:23:56.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:56 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:56.611+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:56.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:56 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:56.611+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:56.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:56 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:56.611+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:56.880 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:56 smithi092 podman[195558]: 2024-09-11 22:23:56.61345599 +0000 UTC m=+5.599359502 container died de39e20a52f0b6eadb7ceeab2d39fc1ec4b706fd7620a14e53edf3ea0fb54d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T22:23:56.880 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:56 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:56.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:56.880 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:56 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:56.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:56.880 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:56 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:56.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:56.880 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:56 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:56.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:23:57.246 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:56 smithi092 podman[195558]: 2024-09-11 22:23:56.978883372 +0000 UTC m=+5.964786891 container cleanup de39e20a52f0b6eadb7ceeab2d39fc1ec4b706fd7620a14e53edf3ea0fb54d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T22:23:57.247 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:56 smithi092 bash[195558]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6 2024-09-11T22:23:57.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:57.331+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:57.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:57.331+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:57.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:57.331+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:23:57.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:57.103+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:57.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:57.103+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:57.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:57.103+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:57.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:57 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:57.103+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:23:57.518 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:57 smithi092 podman[195573]: 2024-09-11 22:23:57.246882627 +0000 UTC m=+0.632137444 container remove de39e20a52f0b6eadb7ceeab2d39fc1ec4b706fd7620a14e53edf3ea0fb54d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T22:23:57.662 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:57 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:57.371+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:57.663 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:57 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:57.371+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:57.663 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:57 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:57.371+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:57.663 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:57 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:57.371+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:23:57.859 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:57.833+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:57.859 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:57.833+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:57.859 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:57.833+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:57.859 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:57 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:57.833+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:23:57.859 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:57 smithi092 podman[195660]: 2024-09-11 22:23:57.496952837 +0000 UTC m=+0.015407531 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:23:57.860 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:57 smithi092 podman[195660]: 2024-09-11 22:23:57.631568486 +0000 UTC m=+0.150023249 container create e7b1d8c30e6f0bc19163970e5abe7d954116da699720047cf3ab9a629abf28d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T22:23:57.860 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:57 smithi092 podman[195660]: 2024-09-11 22:23:57.85944493 +0000 UTC m=+0.377899629 container init e7b1d8c30e6f0bc19163970e5abe7d954116da699720047cf3ab9a629abf28d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T22:23:58.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:57 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:57.662+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:58.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:57 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:57.662+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:58.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:57 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:57.662+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:58.268 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:57 smithi092 podman[195660]: 2024-09-11 22:23:57.862826405 +0000 UTC m=+0.381281106 container start e7b1d8c30e6f0bc19163970e5abe7d954116da699720047cf3ab9a629abf28d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T22:23:58.269 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:57 smithi092 podman[195660]: 2024-09-11 22:23:57.934245663 +0000 UTC m=+0.452700354 container attach e7b1d8c30e6f0bc19163970e5abe7d954116da699720047cf3ab9a629abf28d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T22:23:58.269 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 11 22:23:58 smithi092 podman[195660]: 2024-09-11 22:23:58.054373005 +0000 UTC m=+0.572827705 container died e7b1d8c30e6f0bc19163970e5abe7d954116da699720047cf3ab9a629abf28d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:23:58.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:58 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:58.311+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:58.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:58 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:58.311+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:58.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:58 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:58.311+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:23:58.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:58 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:58.121+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:58.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:58 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:58.121+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:58.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:58 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:58.121+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:58.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:58 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:58.121+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:23:58.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T22:23:58.660 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:58.359+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:58.660 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:58.359+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:58.660 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:58.359+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:58.660 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:58.359+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:23:58.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:23:58.710 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T22:23:58.711 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T22:23:58.711 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.7 2024-09-11T22:23:59.009 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:58 smithi110 systemd[1]: Stopping Ceph osd.7 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:23:59.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:58 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:58.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:23:59.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:58 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:58.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:23:59.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:58 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:58.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:23:59.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:58 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:58.828+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:23:59.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:58.660+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:23:59.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:58.660+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:23:59.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:58 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:58.660+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:23:59.288 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:59.008+0000 7fb5a7c59640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:59.008+0000 7fb5a7c59640 -1 osd.7 98 *** Got signal Terminated *** 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:59.008+0000 7fb5a7c59640 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:59.287+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:59.287+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:23:59.287+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:59.107+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:59.107+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:59.107+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:23:59.289 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:23:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:23:59.107+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:23:59.628 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:59 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:59.321+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:23:59.629 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:59 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:59.321+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:23:59.629 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:59 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:59.321+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:23:59.629 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:23:59 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:23:59.321+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:00.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:59 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:59.628+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:00.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:59 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:59.628+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:00.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:23:59 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:23:59.628+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:00.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:59 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:59.781+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:00.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:59 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:59.781+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:00.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:59 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:59.781+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:00.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:23:59 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:23:59.781+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:00.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:00.281+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:24:00.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:00.281+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:24:00.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:00.281+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:24:00.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:00.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:00.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:00.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:00.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:00.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:00.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:00 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:00.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:00.584 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:00 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:00.319+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:00.584 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:00 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:00.319+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:00.585 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:00 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:00.319+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:00.585 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:00 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:00.319+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:01.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:00 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:00.584+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:01.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:00 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:00.584+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:01.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:00 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:00.584+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:01.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:00 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:00.818+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:01.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:00 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:00.818+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:01.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:00 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:00.818+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:01.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:00 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:00.818+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:01.369 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:01 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:01.246+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:24:01.370 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:01 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:01.247+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:24:01.370 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:01 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:01.247+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:24:01.370 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:01 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:01.045+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:01.370 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:01 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:01.045+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:01.370 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:01 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:01.045+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:01.370 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:01 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:01.045+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:01.587 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:01 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:01.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:01.588 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:01 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:01.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:01.588 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:01 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:01.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:01.588 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:01 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:01.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:02.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:01 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:01.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:02.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:01 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:01.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:02.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:01 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:01.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:02.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:01 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:01.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:02.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:01 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:01.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:02.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:01 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:01.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:02.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:01 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:01.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:02.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:02 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:02.013+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:02.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:02 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:02.013+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:02.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:02 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:02.013+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:02.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:02 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:02.013+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:02.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:02 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:02.204+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:24:02.468 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:02 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:02.204+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:24:02.468 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:02 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:02.204+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:24:02.579 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:02 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:02.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:02.579 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:02 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:02.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:02.579 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:02 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:02.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:02.579 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:02 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:02.292+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:03.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:02 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:02.578+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:03.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:02 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:02.578+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:03.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:02 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:02.578+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:03.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:02 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:02.865+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:03.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:02 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:02.865+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:03.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:02 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:02.865+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:03.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:02 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:02.865+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:03.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:02 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:02.865+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:03.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:03.183+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:13.925431+0000 front 2024-09-11T22:23:13.925551+0000 (oldest deadline 2024-09-11T22:23:38.625535+0000) 2024-09-11T22:24:03.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:03.183+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:18.626114+0000 front 2024-09-11T22:23:18.626178+0000 (oldest deadline 2024-09-11T22:23:48.026624+0000) 2024-09-11T22:24:03.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7[52731]: 2024-09-11T22:24:03.183+0000 7fb5a0267640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.027105+0000 front 2024-09-11T22:23:28.027155+0000 (oldest deadline 2024-09-11T22:23:53.927319+0000) 2024-09-11T22:24:03.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:03.043+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:03.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:03.043+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:03.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:03.043+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:03.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:03.043+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:03.536 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:03 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:03.283+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:03.536 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:03 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:03.283+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:03.537 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:03 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:03.283+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:03.537 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:03 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:03.283+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:03.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:03 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:03.536+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:03.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:03 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:03.536+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:03.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:03 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:03.536+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:04.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:03.851+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:04.408 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:03.851+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:04.409 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:03.851+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:04.409 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:03.851+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:04.409 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:03 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:03.851+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:04.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:04 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:04.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:04.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:04 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:04.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:04.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:04 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:04.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:04.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:04 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:04.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:04.771 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:04.533+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:04.771 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:04.533+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:04.771 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:04.533+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:04.771 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:04.533+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:04.772 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:04.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:04.772 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:04.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:04.772 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:04.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:04.772 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:04 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:04.315+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:04.784 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:04 smithi110 podman[70036]: 2024-09-11 22:24:04.496837708 +0000 UTC m=+5.583014918 container died 45265d510f721f8dbcb219b2a9030c2f4ab72066f446ee26c1bc9e9d8a0af500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T22:24:05.083 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:04 smithi110 podman[70036]: 2024-09-11 22:24:04.884051042 +0000 UTC m=+5.970228251 container cleanup 45265d510f721f8dbcb219b2a9030c2f4ab72066f446ee26c1bc9e9d8a0af500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T22:24:05.084 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:04 smithi110 bash[70036]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7 2024-09-11T22:24:05.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:04 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:04.864+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:05.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:04 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:04.864+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:05.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:04 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:04.864+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:05.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:04 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:04.864+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:05.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:04 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:04.864+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:05.467 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:05 smithi110 podman[70062]: 2024-09-11 22:24:05.177426814 +0000 UTC m=+0.677423229 container remove 45265d510f721f8dbcb219b2a9030c2f4ab72066f446ee26c1bc9e9d8a0af500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T22:24:05.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:05 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:05.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:05.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:05 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:05.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:05.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:05 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:05.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:05.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:05 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:05.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:05.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:05 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:05.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:05.780 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:05 smithi110 podman[70150]: 2024-09-11 22:24:05.407506989 +0000 UTC m=+0.015846080 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:24:05.780 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:05 smithi110 podman[70150]: 2024-09-11 22:24:05.536599718 +0000 UTC m=+0.144938801 container create 73a6c6656d31bf3232ed0711ef5b48f7c4c7f62effd1ed788a1996577e1b4b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T22:24:05.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:05.360+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:05.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:05.360+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:05.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:05.360+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:05.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:05.360+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:05.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:05.539+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:05.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:05.539+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:05.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:05.539+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:05.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:05 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:05.539+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:06.075 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:05 smithi110 podman[70150]: 2024-09-11 22:24:05.780524529 +0000 UTC m=+0.388863620 container init 73a6c6656d31bf3232ed0711ef5b48f7c4c7f62effd1ed788a1996577e1b4b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T22:24:06.075 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:05 smithi110 podman[70150]: 2024-09-11 22:24:05.784023685 +0000 UTC m=+0.392362770 container start 73a6c6656d31bf3232ed0711ef5b48f7c4c7f62effd1ed788a1996577e1b4b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-11T22:24:06.075 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:05 smithi110 podman[70150]: 2024-09-11 22:24:05.880730223 +0000 UTC m=+0.489069319 container attach 73a6c6656d31bf3232ed0711ef5b48f7c4c7f62effd1ed788a1996577e1b4b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T22:24:06.075 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 22:24:05 smithi110 podman[70150]: 2024-09-11 22:24:05.971722508 +0000 UTC m=+0.580061593 container died 73a6c6656d31bf3232ed0711ef5b48f7c4c7f62effd1ed788a1996577e1b4b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-11T22:24:06.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:05 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:05.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:06.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:05 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:05.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:06.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:05 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:05.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:06.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:05 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:05.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:06.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:05 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:05.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:06.367 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:06 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:06.073+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:06.367 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:06 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:06.073+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:06.367 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:06 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:06.073+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:06.367 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:06 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:06.073+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:06.367 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:06 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:06.073+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:06.671 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T22:24:06.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:06.712 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T22:24:06.712 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T22:24:06.712 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.8 2024-09-11T22:24:06.740 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:06.564+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:06.740 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:06.564+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:06.740 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:06.564+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:06.740 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:06.564+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:06.740 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:06.354+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:06.740 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:06.354+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:06.741 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:06.354+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:06.741 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:06 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:06.354+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:07.027 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:06 smithi153 systemd[1]: Stopping Ceph osd.8 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:07.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:06 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:06.843+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:07.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:06 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:06.843+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:07.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:06 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:06.843+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:07.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:06 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:06.843+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:07.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:06 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:06.843+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:07.289 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:07.026+0000 7efe75c2c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:07.289 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:07.026+0000 7efe75c2c640 -1 osd.8 98 *** Got signal Terminated *** 2024-09-11T22:24:07.289 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:07.026+0000 7efe75c2c640 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:24:07.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:07 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:07.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:07.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:07 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:07.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:07.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:07 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:07.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:07.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:07 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:07.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:07.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:07 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:07.082+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:07.595 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:07.328+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:07.595 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:07.328+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:07.595 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:07.328+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:07.595 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:07.328+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:08.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:07.594+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:08.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:07.594+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:08.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:07.594+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:08.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:07 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:07.594+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:08.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:07 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:07.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:08.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:07 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:07.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:08.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:07 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:07.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:08.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:07 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:07.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:08.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:07 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:07.856+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:08.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:08 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:08.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:08.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:08 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:08.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:08.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:08 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:08.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:08.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:08 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:08.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:08.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:08 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:08.054+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:08.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:08.317+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:08.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:08.317+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:08.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:08.317+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:08.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:08.317+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:08.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:08.557+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:08.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:08.557+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:08.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:08.557+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:08.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:08 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:08.557+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:09.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:08 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:08.829+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:09.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:08 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:08.829+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:09.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:08 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:08.829+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:09.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:08 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:08.829+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:09.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:08 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:08.829+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:09.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:09 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:09.032+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:09.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:09 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:09.032+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:09.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:09 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:09.032+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:09.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:09 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:09.032+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:09.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:09 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:09.032+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:09.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:09.282+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:09.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:09.282+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:09.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:09.282+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:09.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:09.282+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:09.539 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:09.509+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:09.539 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:09.509+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:09.539 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:09.509+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:09.540 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:09 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:09.509+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:10.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:09 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:09.858+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:10.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:09 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:09.858+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:10.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:09 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:09.858+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:10.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:09 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:09.858+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:10.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:09 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:09.858+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:10.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:09 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:09.858+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:10.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:10 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:10.066+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:10.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:10 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:10.066+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:10.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:10 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:10.066+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:10.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:10 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:10.066+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:10.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:10 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:10.066+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:10.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:10 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:10.066+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:10.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:10.296+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:10.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:10.296+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:10.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:10.296+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:10.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:10.296+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:10.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:10.483+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:10.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:10.483+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:10.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:10.483+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:10.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:10 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:10.483+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:11.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:10.822+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:11.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:10.822+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:11.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:10.822+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:11.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:10.822+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:11.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:10.822+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:11.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:10 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:10.822+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:11.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:11.062+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:11.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:11.062+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:11.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:11.062+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:11.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:11.062+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:11.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:11.062+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:11.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:11 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:11.062+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:11.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:11.280+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.587856+0000 front 2024-09-11T22:23:06.587707+0000 (oldest deadline 2024-09-11T22:23:32.487941+0000) 2024-09-11T22:24:11.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:11.280+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.488524+0000 front 2024-09-11T22:23:12.488641+0000 (oldest deadline 2024-09-11T22:23:38.388526+0000) 2024-09-11T22:24:11.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:11.280+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:24.190080+0000 front 2024-09-11T22:23:24.190095+0000 (oldest deadline 2024-09-11T22:23:48.890207+0000) 2024-09-11T22:24:11.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:11.280+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:28.890753+0000 front 2024-09-11T22:23:28.890643+0000 (oldest deadline 2024-09-11T22:23:54.190536+0000) 2024-09-11T22:24:11.539 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8[53614]: 2024-09-11T22:24:11.280+0000 7efe6e23a640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.692289+0000 front 2024-09-11T22:23:47.692249+0000 (oldest deadline 2024-09-11T22:24:11.192509+0000) 2024-09-11T22:24:11.539 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:11.496+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:11.540 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:11.496+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:11.540 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:11.496+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:11.540 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:11.496+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:11.540 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:11.496+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:12.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:11.852+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:12.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:11.852+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:12.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:11.852+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:12.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:11.852+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:12.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:11.852+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:12.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:11 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:11.852+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:12.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.025+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:12.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.025+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:12.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.025+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:12.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.025+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:12.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.025+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:12.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.025+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:12.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:12 smithi153 podman[70315]: 2024-09-11 22:24:12.522860236 +0000 UTC m=+5.616092123 container died 9c33413ac49e431a56502be1015daa1d1a850121d12c6bf95b0dfa0ff7134ff1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T22:24:12.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:12.450+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:12.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:12.450+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:12.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:12.450+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:12.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:12.450+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:12.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:12 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:12.450+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:13.170 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:12 smithi153 podman[70315]: 2024-09-11 22:24:12.918835889 +0000 UTC m=+6.012067774 container cleanup 9c33413ac49e431a56502be1015daa1d1a850121d12c6bf95b0dfa0ff7134ff1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:24:13.170 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:12 smithi153 bash[70315]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8 2024-09-11T22:24:13.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:12.831+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:13.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:12.831+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:13.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:12.831+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:13.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:12.831+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:13.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:12.831+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:13.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:12 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:12.831+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:13.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.990+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:13.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.990+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:13.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.990+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:13.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.990+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:13.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.990+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:13.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:12 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:12.990+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:13.530 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:13 smithi153 podman[70330]: 2024-09-11 22:24:13.170097734 +0000 UTC m=+0.637470292 container remove 9c33413ac49e431a56502be1015daa1d1a850121d12c6bf95b0dfa0ff7134ff1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T22:24:13.531 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:13.406+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:13.531 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:13.406+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:13.531 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:13.406+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:13.531 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:13.406+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:13.531 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:13 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:13.406+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:13.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:13 smithi153 podman[70422]: 2024-09-11 22:24:13.433311141 +0000 UTC m=+0.015771426 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:24:13.789 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:13 smithi153 podman[70422]: 2024-09-11 22:24:13.562865397 +0000 UTC m=+0.145325730 container create 370cb4b173ace22f0f766562cb01eb2f0d38a6a3a133440a62244df112f7aecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T22:24:14.113 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:13 smithi153 podman[70422]: 2024-09-11 22:24:13.923599332 +0000 UTC m=+0.506059619 container init 370cb4b173ace22f0f766562cb01eb2f0d38a6a3a133440a62244df112f7aecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T22:24:14.113 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:13 smithi153 podman[70422]: 2024-09-11 22:24:13.92674982 +0000 UTC m=+0.509210096 container start 370cb4b173ace22f0f766562cb01eb2f0d38a6a3a133440a62244df112f7aecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T22:24:14.113 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:14 smithi153 podman[70422]: 2024-09-11 22:24:14.006977839 +0000 UTC m=+0.589438128 container attach 370cb4b173ace22f0f766562cb01eb2f0d38a6a3a133440a62244df112f7aecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T22:24:14.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:13.849+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:14.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:13.849+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:14.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:13.849+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:14.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:13.849+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:14.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:13.849+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:14.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:13 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:13.849+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:14.417 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 11 22:24:14 smithi153 podman[70422]: 2024-09-11 22:24:14.112414468 +0000 UTC m=+0.694874752 container died 370cb4b173ace22f0f766562cb01eb2f0d38a6a3a133440a62244df112f7aecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T22:24:14.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.021+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:14.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.021+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:14.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.021+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:14.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.021+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:14.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.021+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:14.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.021+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:14.731 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:14.415+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:14.731 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:14.415+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:14.731 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:14.415+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:14.731 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:14.415+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:14.731 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:14 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:14.415+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:14.746 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T22:24:14.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:14.826 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T22:24:14.826 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T22:24:14.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.9 2024-09-11T22:24:15.141 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:14 smithi092 systemd[1]: Stopping Ceph osd.9 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:15.141 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:14.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:15.141 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:14.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:15.141 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:14.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:15.141 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:14.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:15.142 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:14.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:15.142 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:14 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:14.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:15.142 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.138+0000 7fb4d2008640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:15.142 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.138+0000 7fb4d2008640 -1 osd.9 98 *** Got signal Terminated *** 2024-09-11T22:24:15.142 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.138+0000 7fb4d2008640 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:24:15.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.998+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:15.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.998+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:15.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.998+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:15.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.998+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:15.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.998+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:15.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:14 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:14.998+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:15.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:15.395+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:15.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:15.395+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:15.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:15.395+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:15.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:15.395+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:15.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:15.395+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:16.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:16.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:16.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:16.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:16.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:16.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:15 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:15.877+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:16.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:16.037+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:16.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:16.037+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:16.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:16.037+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:16.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:16.037+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:16.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:16.037+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:16.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:16 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:16.037+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:16.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:16 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:16.363+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:16.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:16 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:16.363+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:16.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:16 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:16.363+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:16.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:16 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:16.363+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:16.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:16 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:16.363+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:17.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:16.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:17.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:16.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:17.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:16.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:17.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:16.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:17.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:16.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:17.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:16 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:16.882+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:17.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:17 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:17.080+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:17.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:17 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:17.080+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:17.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:17 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:17.080+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:17.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:17 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:17.080+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:17.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:17 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:17.080+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:17.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:17 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:17.080+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:17.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:17 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:17.398+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:17.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:17 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:17.398+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:17.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:17 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:17.398+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:17.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:17 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:17.398+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:17.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:17 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:17.398+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:17.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:17 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:17.398+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:18.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:17 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:17.846+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:18.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:17 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:17.846+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:18.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:17 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:17.846+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:18.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:17 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:17.846+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:18.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:17 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:17.846+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:18.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:17 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:17.846+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:18.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:18 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:18.100+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:18.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:18 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:18.100+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:18.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:18 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:18.100+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:18.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:18 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:18.100+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:18.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:18 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:18.100+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:18.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:18 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:18.100+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:18.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:18 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:18.437+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:18.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:18 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:18.437+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:18.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:18 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:18.437+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:18.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:18 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:18.437+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:18.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:18 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:18.437+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:18.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:18 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:18.437+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:19.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:18 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:18.830+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:19.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:18 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:18.830+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:19.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:18 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:18.830+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:19.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:18 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:18.830+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:19.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:18 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:18.830+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:19.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:18 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:18.830+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:19.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:19 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:19.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:19.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:19 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:19.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:19.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:19 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:19.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:19.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:19 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:19.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:19.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:19 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:19.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:19.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:19 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:19.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:19.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:19.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:19.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:19.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:19.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:19.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:19.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:19.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:19.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:19.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:19.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:19.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:20.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:19 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:19.806+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:03.412303+0000 front 2024-09-11T22:23:03.412284+0000 (oldest deadline 2024-09-11T22:23:29.312523+0000) 2024-09-11T22:24:20.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:19 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:19.806+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.813849+0000 front 2024-09-11T22:23:12.813909+0000 (oldest deadline 2024-09-11T22:23:38.713681+0000) 2024-09-11T22:24:20.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:19 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:19.806+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.814778+0000 front 2024-09-11T22:23:19.815463+0000 (oldest deadline 2024-09-11T22:23:48.015242+0000) 2024-09-11T22:24:20.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:19 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:19.806+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:31.515864+0000 front 2024-09-11T22:23:31.515834+0000 (oldest deadline 2024-09-11T22:23:55.015966+0000) 2024-09-11T22:24:20.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:19 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:19.806+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:35.516987+0000 front 2024-09-11T22:23:35.516974+0000 (oldest deadline 2024-09-11T22:24:02.517105+0000) 2024-09-11T22:24:20.269 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:19 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9[65874]: 2024-09-11T22:24:19.806+0000 7fb4ca616640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:46.617974+0000 front 2024-09-11T22:23:46.618051+0000 (oldest deadline 2024-09-11T22:24:08.917819+0000) 2024-09-11T22:24:20.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:20 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:20.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:20.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:20 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:20.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:20.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:20 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:20.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:20.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:20 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:20.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:20.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:20 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:20.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:20.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:20 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:20.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:20.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:20 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:20.070+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:20.768 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:20 smithi092 podman[195777]: 2024-09-11 22:24:20.512738676 +0000 UTC m=+5.496742431 container died 49b82df131f1b2dccf1073f988bd509d3653b63735903214db6ff422ca45c2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T22:24:20.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:20 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:20.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:20.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:20 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:20.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:20.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:20 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:20.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:20.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:20 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:20.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:20.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:20 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:20.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:20.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:20 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:20.445+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:21.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:20 smithi092 podman[195777]: 2024-09-11 22:24:20.855104943 +0000 UTC m=+5.839108708 container cleanup 49b82df131f1b2dccf1073f988bd509d3653b63735903214db6ff422ca45c2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T22:24:21.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:20 smithi092 bash[195777]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9 2024-09-11T22:24:21.268 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:21 smithi092 podman[195794]: 2024-09-11 22:24:21.031149484 +0000 UTC m=+0.519484901 container remove 49b82df131f1b2dccf1073f988bd509d3653b63735903214db6ff422ca45c2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T22:24:21.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:21 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:21.039+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:21.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:21 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:21.039+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:21.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:21 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:21.039+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:21.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:21 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:21.039+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:21.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:21 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:21.039+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:21.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:21 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:21.039+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:21.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:21 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:21.039+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:21.751 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:21 smithi092 podman[195884]: 2024-09-11 22:24:21.365404267 +0000 UTC m=+0.015953775 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:24:21.751 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:21 smithi092 podman[195884]: 2024-09-11 22:24:21.465258664 +0000 UTC m=+0.115808204 container create df24d257ad4aaef19457e1c0f5e108d25820b95a5e19b85d8462f9909f62704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:24:21.751 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:21 smithi092 podman[195884]: 2024-09-11 22:24:21.676124525 +0000 UTC m=+0.326674019 container init df24d257ad4aaef19457e1c0f5e108d25820b95a5e19b85d8462f9909f62704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T22:24:21.751 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:21 smithi092 podman[195884]: 2024-09-11 22:24:21.679485518 +0000 UTC m=+0.330035023 container start df24d257ad4aaef19457e1c0f5e108d25820b95a5e19b85d8462f9909f62704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T22:24:21.751 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:21 smithi092 podman[195884]: 2024-09-11 22:24:21.751098124 +0000 UTC m=+0.401647628 container attach df24d257ad4aaef19457e1c0f5e108d25820b95a5e19b85d8462f9909f62704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:24:21.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:21 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:21.462+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:21.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:21 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:21.462+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:21.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:21 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:21.462+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:21.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:21 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:21.462+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:21.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:21 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:21.462+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:21.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:21 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:21.462+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:22.018 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:21 smithi092 podman[195884]: 2024-09-11 22:24:21.868384943 +0000 UTC m=+0.518934448 container died df24d257ad4aaef19457e1c0f5e108d25820b95a5e19b85d8462f9909f62704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T22:24:22.364 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 11 22:24:22 smithi092 podman[195907]: 2024-09-11 22:24:22.346073226 +0000 UTC m=+0.469165466 container remove df24d257ad4aaef19457e1c0f5e108d25820b95a5e19b85d8462f9909f62704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-11T22:24:22.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T22:24:22.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:22.410 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T22:24:22.410 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T22:24:22.410 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.10 2024-09-11T22:24:22.437 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.023+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:22.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.023+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:22.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.023+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:22.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.023+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:22.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.023+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:22.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.023+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:22.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.023+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:22.712 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 systemd[1]: Stopping Ceph osd.10 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:22.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:22 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:22.459+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:22.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:22 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:22.459+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:22.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:22 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:22.459+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:22.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:22 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:22.459+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:22.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:22 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:22.459+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:22.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:22 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:22.459+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:22.967 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.711+0000 7fdc2ce5a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:22.967 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.711+0000 7fdc2ce5a640 -1 osd.10 98 *** Got signal Terminated *** 2024-09-11T22:24:22.967 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:22 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:22.711+0000 7fdc2ce5a640 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:24:23.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:23 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:23.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:23.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:23 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:23.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:23.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:23 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:23.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:23.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:23 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:23.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:23.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:23 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:23.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:23.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:23 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:23.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:23.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:23 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:23.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:23.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:23.479+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:23.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:23.479+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:23.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:23.479+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:23.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:23.479+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:23.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:23.479+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:23.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:23.479+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:24.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:24 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:24.052+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:24.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:24 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:24.053+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:24.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:24 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:24.053+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:24.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:24 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:24.053+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:24.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:24 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:24.053+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:24.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:24 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:24.053+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:24.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:24 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:24.053+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:24.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:24 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:24.455+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:24.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:24 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:24.455+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:24.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:24 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:24.455+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:24.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:24 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:24.455+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:24.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:24 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:24.455+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:24.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:24 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:24.455+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:25.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:25 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:25.009+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:25.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:25 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:25.009+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:25.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:25 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:25.009+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:25.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:25 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:25.009+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:25.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:25 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:25.009+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:25.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:25 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:25.009+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:25.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:25 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:25.009+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:25.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:25 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:25.478+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:25.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:25 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:25.478+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:25.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:25 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:25.478+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:25.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:25 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:25.478+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:25.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:25 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:25.478+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:25.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:25 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:25.478+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:26.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.004+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:26.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.004+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:26.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.004+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:26.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.004+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:26.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.004+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:26.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.004+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:26.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.004+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:26.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:26.499+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:26.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:26.499+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:26.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:26.499+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:26.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:26.499+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:26.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:26.499+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:26.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:26.499+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:26.790 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:26 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:26.499+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:27.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.970+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.696466+0000 front 2024-09-11T22:23:06.696455+0000 (oldest deadline 2024-09-11T22:23:30.196571+0000) 2024-09-11T22:24:27.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.970+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.998115+0000 front 2024-09-11T22:23:12.998212+0000 (oldest deadline 2024-09-11T22:23:38.298279+0000) 2024-09-11T22:24:27.467 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.970+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:21.199348+0000 front 2024-09-11T22:23:21.199386+0000 (oldest deadline 2024-09-11T22:23:46.399828+0000) 2024-09-11T22:24:27.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.970+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-11T22:23:32.201823+0000 front 2024-09-11T22:23:32.201747+0000 (oldest deadline 2024-09-11T22:23:53.301752+0000) 2024-09-11T22:24:27.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.970+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:36.703219+0000 front 2024-09-11T22:23:36.703240+0000 (oldest deadline 2024-09-11T22:24:04.303778+0000) 2024-09-11T22:24:27.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.970+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:45.404849+0000 front 2024-09-11T22:23:45.404789+0000 (oldest deadline 2024-09-11T22:24:09.504742+0000) 2024-09-11T22:24:27.468 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:26 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10[59702]: 2024-09-11T22:24:26.970+0000 7fdc29470640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:53.005793+0000 front 2024-09-11T22:23:53.005833+0000 (oldest deadline 2024-09-11T22:24:20.006075+0000) 2024-09-11T22:24:27.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:27 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:27.527+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:27.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:27 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:27.527+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:27.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:27 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:27.527+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:27.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:27 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:27.527+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:27.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:27 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:27.527+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:27.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:27 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:27.527+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:27.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:27 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:27.527+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:28.321 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:28 smithi110 podman[70253]: 2024-09-11 22:24:28.063374097 +0000 UTC m=+5.459468755 container died 342dc30183da37fa034281df381344dde17773597c4db2edd4298f4be00d85be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T22:24:28.646 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:28 smithi110 podman[70253]: 2024-09-11 22:24:28.412163331 +0000 UTC m=+5.808257997 container cleanup 342dc30183da37fa034281df381344dde17773597c4db2edd4298f4be00d85be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T22:24:28.646 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:28 smithi110 bash[70253]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10 2024-09-11T22:24:28.967 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:28 smithi110 podman[70268]: 2024-09-11 22:24:28.646284545 +0000 UTC m=+0.585450659 container remove 342dc30183da37fa034281df381344dde17773597c4db2edd4298f4be00d85be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:24:29.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:28 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:28.560+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:29.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:28 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:28.560+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:29.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:28 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:28.560+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:29.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:28 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:28.560+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:29.040 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:28 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:28.560+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:29.040 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:28 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:28.560+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:29.040 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:28 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:28.560+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:29.274 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:28 smithi110 podman[70355]: 2024-09-11 22:24:28.887829902 +0000 UTC m=+0.015807167 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:24:29.274 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:28 smithi110 podman[70355]: 2024-09-11 22:24:28.988673724 +0000 UTC m=+0.116651021 container create ce2726305ddeb496ed8e980cf3903a93c20e99c12d826b1b6084cbcb57882b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:24:29.274 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:29 smithi110 podman[70355]: 2024-09-11 22:24:29.191119559 +0000 UTC m=+0.319096814 container init ce2726305ddeb496ed8e980cf3903a93c20e99c12d826b1b6084cbcb57882b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:24:29.275 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:29 smithi110 podman[70355]: 2024-09-11 22:24:29.194310962 +0000 UTC m=+0.322288215 container start ce2726305ddeb496ed8e980cf3903a93c20e99c12d826b1b6084cbcb57882b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T22:24:29.275 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:29 smithi110 podman[70355]: 2024-09-11 22:24:29.274386912 +0000 UTC m=+0.402364183 container attach ce2726305ddeb496ed8e980cf3903a93c20e99c12d826b1b6084cbcb57882b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:24:29.677 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 22:24:29 smithi110 podman[70355]: 2024-09-11 22:24:29.377514932 +0000 UTC m=+0.505492199 container died ce2726305ddeb496ed8e980cf3903a93c20e99c12d826b1b6084cbcb57882b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T22:24:29.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T22:24:30.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:30.028 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T22:24:30.029 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T22:24:30.029 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.11 2024-09-11T22:24:30.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:29 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:29.573+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:30.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:29 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:29.573+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:30.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:29 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:29.573+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:30.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:29 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:29.573+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:30.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:29 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:29.573+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:30.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:29 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:29.573+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:30.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:29 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:29.573+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:30.347 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 systemd[1]: Stopping Ceph osd.11 for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:30.347 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.346+0000 7f07659d6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:30.347 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.346+0000 7f07659d6640 -1 osd.11 98 *** Got signal Terminated *** 2024-09-11T22:24:30.347 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.346+0000 7f07659d6640 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:24:30.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.566+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:30.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.566+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:30.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.566+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:30.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.566+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:30.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.566+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:30.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.566+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:30.789 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:30 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:30.566+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:32.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:31 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:31.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:32.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:31 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:31.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:32.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:31 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:31.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:32.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:31 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:31.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:32.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:31 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:31.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:32.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:31 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:31.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:32.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:31 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:31.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:33.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:32 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:32.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:33.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:32 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:32.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:33.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:32 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:32.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:33.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:32 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:32.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:33.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:32 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:32.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:33.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:32 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:32.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:33.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:32 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:32.587+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:33.940 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:33.940 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:33.940 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:33.941 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:33.941 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:33.941 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:33.941 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:33.941 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:33 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:33.616+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6820 osd.8 since back 2024-09-11T22:24:11.530073+0000 front 2024-09-11T22:24:11.530103+0000 (oldest deadline 2024-09-11T22:24:32.630122+0000) 2024-09-11T22:24:35.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T22:23:06.718584+0000 front 2024-09-11T22:23:06.718543+0000 (oldest deadline 2024-09-11T22:23:32.618753+0000) 2024-09-11T22:24:35.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T22:23:12.619309+0000 front 2024-09-11T22:23:12.619459+0000 (oldest deadline 2024-09-11T22:23:37.319296+0000) 2024-09-11T22:24:35.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-11T22:23:19.020489+0000 front 2024-09-11T22:23:19.020383+0000 (oldest deadline 2024-09-11T22:23:44.920622+0000) 2024-09-11T22:24:35.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T22:23:39.824258+0000 front 2024-09-11T22:23:39.824192+0000 (oldest deadline 2024-09-11T22:24:04.524144+0000) 2024-09-11T22:24:35.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-11T22:23:47.325803+0000 front 2024-09-11T22:23:47.325827+0000 (oldest deadline 2024-09-11T22:24:11.425884+0000) 2024-09-11T22:24:35.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-11T22:23:55.427807+0000 front 2024-09-11T22:23:55.427883+0000 (oldest deadline 2024-09-11T22:24:17.127473+0000) 2024-09-11T22:24:35.039 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T22:24:00.028328+0000 front 2024-09-11T22:24:00.028361+0000 (oldest deadline 2024-09-11T22:24:25.928334+0000) 2024-09-11T22:24:35.040 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:34 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11[60117]: 2024-09-11T22:24:34.593+0000 7f0761fec640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.153:6820 osd.8 since back 2024-09-11T22:24:11.530073+0000 front 2024-09-11T22:24:11.530103+0000 (oldest deadline 2024-09-11T22:24:32.630122+0000) 2024-09-11T22:24:35.952 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:35 smithi153 podman[70525]: 2024-09-11 22:24:35.690563899 +0000 UTC m=+5.465854683 container died 7b7c1e39f23476313785bcd54778bb293496796643f174911d30d4ab18341d61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:24:36.222 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 podman[70525]: 2024-09-11 22:24:36.020981779 +0000 UTC m=+5.796272564 container cleanup 7b7c1e39f23476313785bcd54778bb293496796643f174911d30d4ab18341d61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T22:24:36.222 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 bash[70525]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11 2024-09-11T22:24:36.539 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 podman[70541]: 2024-09-11 22:24:36.222339145 +0000 UTC m=+0.530707625 container remove 7b7c1e39f23476313785bcd54778bb293496796643f174911d30d4ab18341d61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T22:24:36.942 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 podman[70630]: 2024-09-11 22:24:36.530210675 +0000 UTC m=+0.015973254 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:24:36.942 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 podman[70630]: 2024-09-11 22:24:36.63980025 +0000 UTC m=+0.125562825 container create 6c7466b656f66979c833fcfe06ea4c00f78aedad08de6eacc6a974de3590aeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:24:36.942 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 podman[70630]: 2024-09-11 22:24:36.858678954 +0000 UTC m=+0.344441526 container init 6c7466b656f66979c833fcfe06ea4c00f78aedad08de6eacc6a974de3590aeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T22:24:36.942 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 podman[70630]: 2024-09-11 22:24:36.861991092 +0000 UTC m=+0.347753667 container start 6c7466b656f66979c833fcfe06ea4c00f78aedad08de6eacc6a974de3590aeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T22:24:36.942 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:36 smithi153 podman[70630]: 2024-09-11 22:24:36.941849808 +0000 UTC m=+0.427612393 container attach 6c7466b656f66979c833fcfe06ea4c00f78aedad08de6eacc6a974de3590aeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:24:37.289 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 11 22:24:37 smithi153 podman[70630]: 2024-09-11 22:24:37.052972302 +0000 UTC m=+0.538734877 container died 6c7466b656f66979c833fcfe06ea4c00f78aedad08de6eacc6a974de3590aeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:24:37.618 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T22:24:37.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:37.697 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T22:24:37.697 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T22:24:37.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.a 2024-09-11T22:24:37.998 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:24:37 smithi092 systemd[1]: Stopping Ceph mds.a for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:38.268 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:24:37 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a[72945]: 2024-09-11T22:24:37.996+0000 7f18b23ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:38.268 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:24:37 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a[72945]: 2024-09-11T22:24:37.996+0000 7f18b23ec640 -1 mds.a *** got signal Terminated *** 2024-09-11T22:24:42.268 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:24:42 smithi092 podman[195999]: 2024-09-11 22:24:42.002571757 +0000 UTC m=+4.110234863 container died 77b3d4333b9def4ec8aae755c8798b262e4e90bcfb494dcb9ccb0bae7e72a66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T22:24:42.614 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:24:42 smithi092 podman[195999]: 2024-09-11 22:24:42.362838829 +0000 UTC m=+4.470501944 container cleanup 77b3d4333b9def4ec8aae755c8798b262e4e90bcfb494dcb9ccb0bae7e72a66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T22:24:42.614 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 11 22:24:42 smithi092 bash[195999]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-a 2024-09-11T22:24:42.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T22:24:42.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:42.786 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T22:24:42.786 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T22:24:42.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.d 2024-09-11T22:24:43.268 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:42 smithi092 systemd[1]: Stopping Ceph mds.d for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:43.268 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:43 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d[71765]: 2024-09-11T22:24:43.090+0000 7fe287a44640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:43.269 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:43 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d[71765]: 2024-09-11T22:24:43.090+0000 7fe287a44640 -1 mds.d *** got signal Terminated *** 2024-09-11T22:24:47.518 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:47 smithi092 podman[196119]: 2024-09-11 22:24:47.09810714 +0000 UTC m=+4.097214120 container died 6576448ede5a1f3b70b9642ddbe6b7c73079ffabb37959179b47a270d6434128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T22:24:47.933 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:47 smithi092 podman[196119]: 2024-09-11 22:24:47.732452377 +0000 UTC m=+4.731559349 container cleanup 6576448ede5a1f3b70b9642ddbe6b7c73079ffabb37959179b47a270d6434128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T22:24:47.934 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:47 smithi092 bash[196119]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d 2024-09-11T22:24:48.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T22:24:48.190 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:47 smithi092 podman[196133]: 2024-09-11 22:24:47.933668422 +0000 UTC m=+0.826219392 container remove 6576448ede5a1f3b70b9642ddbe6b7c73079ffabb37959179b47a270d6434128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T22:24:48.190 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:48 smithi092 systemd[1]: ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-11T22:24:48.190 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 11 22:24:48 smithi092 systemd[1]: Stopped Ceph mds.d for a6709e28-7089-11ef-bcea-c7b262605968. 2024-09-11T22:24:48.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:48.202 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T22:24:48.202 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T22:24:48.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.g 2024-09-11T22:24:48.511 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:24:48 smithi092 systemd[1]: Stopping Ceph mds.g for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:48.768 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:24:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g[72562]: 2024-09-11T22:24:48.510+0000 7fa86c957640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:48.768 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:24:48 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g[72562]: 2024-09-11T22:24:48.510+0000 7fa86c957640 -1 mds.g *** got signal Terminated *** 2024-09-11T22:24:50.929 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:24:50 smithi092 podman[196239]: 2024-09-11 22:24:50.671906359 +0000 UTC m=+2.253817539 container died 8f461e252924867d6f65e742a4d7cfe48c06692688f3f792ebe9fda2dc7cfa99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T22:24:51.268 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:24:51 smithi092 podman[196239]: 2024-09-11 22:24:51.044631723 +0000 UTC m=+2.626542892 container cleanup 8f461e252924867d6f65e742a4d7cfe48c06692688f3f792ebe9fda2dc7cfa99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-11T22:24:51.269 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 11 22:24:51 smithi092 bash[196239]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-g 2024-09-11T22:24:51.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T22:24:51.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:51.433 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T22:24:51.433 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T22:24:51.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.j 2024-09-11T22:24:51.723 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:24:51 smithi092 systemd[1]: Stopping Ceph mds.j for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:52.018 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:24:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j[71348]: 2024-09-11T22:24:51.722+0000 7f31042bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:52.018 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:24:51 smithi092 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j[71348]: 2024-09-11T22:24:51.722+0000 7f31042bd640 -1 mds.j *** got signal Terminated *** 2024-09-11T22:24:55.998 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:24:55 smithi092 podman[196361]: 2024-09-11 22:24:55.729442414 +0000 UTC m=+4.084780147 container died 6535b833ed09aa08dd979cfd2477cd8b1b0dfe2424aefb335dee36e41c4e3893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T22:24:56.268 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:24:56 smithi092 podman[196361]: 2024-09-11 22:24:56.0892589 +0000 UTC m=+4.444596628 container cleanup 6535b833ed09aa08dd979cfd2477cd8b1b0dfe2424aefb335dee36e41c4e3893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T22:24:56.268 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 11 22:24:56 smithi092 bash[196361]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-j 2024-09-11T22:24:56.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T22:24:56.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:56.504 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T22:24:56.504 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T22:24:56.504 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.b 2024-09-11T22:24:56.879 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:24:56 smithi110 systemd[1]: Stopping Ceph mds.b for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:24:57.131 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:24:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: 2024-09-11T22:24:56.878+0000 7f957c87c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:24:57.131 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:24:56 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b[65146]: 2024-09-11T22:24:56.878+0000 7f957c87c640 -1 mds.b *** got signal Terminated *** 2024-09-11T22:24:58.967 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:24:58 smithi110 podman[70458]: 2024-09-11 22:24:58.673066283 +0000 UTC m=+1.967477846 container died 94775968d4fb1ca341cef2b01b806bd120bc54e89b447b635b2bd1294a04b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:24:59.467 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:24:59 smithi110 podman[70458]: 2024-09-11 22:24:59.253680232 +0000 UTC m=+2.548091858 container cleanup 94775968d4fb1ca341cef2b01b806bd120bc54e89b447b635b2bd1294a04b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:24:59.467 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 22:24:59 smithi110 bash[70458]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-b 2024-09-11T22:24:59.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T22:24:59.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:24:59.672 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T22:24:59.672 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T22:24:59.673 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.e 2024-09-11T22:24:59.967 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:24:59 smithi110 systemd[1]: Stopping Ceph mds.e for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:25:00.468 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:24:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e[64539]: 2024-09-11T22:24:59.973+0000 7f403c1a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:25:00.885 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:24:59 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e[64539]: 2024-09-11T22:24:59.973+0000 7f403c1a0640 -1 mds.e *** got signal Terminated *** 2024-09-11T22:25:02.967 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:25:02 smithi110 podman[70590]: 2024-09-11 22:25:02.683639938 +0000 UTC m=+2.788888801 container died e28c89905fea9ccf2a95b7d6042cd0daa5b5b16dc4ac72244ec649d4f618064b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-11T22:25:03.410 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:25:02 smithi110 podman[70590]: 2024-09-11 22:25:02.998806437 +0000 UTC m=+3.104055299 container cleanup e28c89905fea9ccf2a95b7d6042cd0daa5b5b16dc4ac72244ec649d4f618064b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T22:25:03.410 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:25:02 smithi110 bash[70590]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e 2024-09-11T22:25:03.410 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 22:25:03 smithi110 podman[70604]: 2024-09-11 22:25:03.216769005 +0000 UTC m=+0.534002988 container remove e28c89905fea9ccf2a95b7d6042cd0daa5b5b16dc4ac72244ec649d4f618064b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T22:25:03.429 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T22:25:03.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:25:03.463 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T22:25:03.463 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T22:25:03.463 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.h 2024-09-11T22:25:03.717 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:25:03 smithi110 systemd[1]: Stopping Ceph mds.h for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:25:04.217 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:25:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h[64844]: 2024-09-11T22:25:03.811+0000 7f0b58c2b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:25:04.217 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:25:03 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h[64844]: 2024-09-11T22:25:03.811+0000 7f0b58c2b640 -1 mds.h *** got signal Terminated *** 2024-09-11T22:25:06.954 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:25:06 smithi110 podman[70710]: 2024-09-11 22:25:06.6794669 +0000 UTC m=+2.998106163 container died 40db97aa53028368176027065cfb826f829d646a20abbc15b25bcca9515472fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:25:07.217 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:25:07 smithi110 podman[70710]: 2024-09-11 22:25:07.027937885 +0000 UTC m=+3.346577136 container cleanup 40db97aa53028368176027065cfb826f829d646a20abbc15b25bcca9515472fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T22:25:07.217 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 22:25:07 smithi110 bash[70710]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-h 2024-09-11T22:25:07.452 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T22:25:07.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:25:07.483 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T22:25:07.484 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T22:25:07.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.k 2024-09-11T22:25:07.856 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:25:07 smithi110 systemd[1]: Stopping Ceph mds.k for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:25:08.159 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:25:07 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k[64227]: 2024-09-11T22:25:07.856+0000 7fac40ae1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:25:08.159 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:25:07 smithi110 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k[64227]: 2024-09-11T22:25:07.856+0000 7fac40ae1640 -1 mds.k *** got signal Terminated *** 2024-09-11T22:25:10.966 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:25:10 smithi110 podman[70830]: 2024-09-11 22:25:10.664085331 +0000 UTC m=+2.955189962 container died de5280b9f56ad3bfba2bb01b827c282925adbefe2443a87b609c6330aedfe026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T22:25:11.217 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:25:11 smithi110 podman[70830]: 2024-09-11 22:25:11.056452847 +0000 UTC m=+3.347557477 container cleanup de5280b9f56ad3bfba2bb01b827c282925adbefe2443a87b609c6330aedfe026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T22:25:11.217 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 22:25:11 smithi110 bash[70830]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-k 2024-09-11T22:25:11.442 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T22:25:11.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:25:11.474 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T22:25:11.474 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T22:25:11.474 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.c 2024-09-11T22:25:11.770 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:25:11 smithi153 systemd[1]: Stopping Ceph mds.c for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:25:12.034 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:25:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c[65369]: 2024-09-11T22:25:11.768+0000 7f367d61d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:25:12.034 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:25:11 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c[65369]: 2024-09-11T22:25:11.768+0000 7f367d61d640 -1 mds.c *** got signal Terminated *** 2024-09-11T22:25:14.961 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:25:14 smithi153 podman[70735]: 2024-09-11 22:25:14.67255895 +0000 UTC m=+3.007807385 container died a592d442b3a63c3b979ea45a57f468b6954df1fb41769dbce2856fecf2033c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T22:25:15.288 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:25:15 smithi153 podman[70735]: 2024-09-11 22:25:15.060289707 +0000 UTC m=+3.395538134 container cleanup a592d442b3a63c3b979ea45a57f468b6954df1fb41769dbce2856fecf2033c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T22:25:15.289 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 11 22:25:15 smithi153 bash[70735]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-c 2024-09-11T22:25:15.443 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T22:25:15.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:25:15.480 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T22:25:15.480 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T22:25:15.480 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.f 2024-09-11T22:25:15.807 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:25:15 smithi153 systemd[1]: Stopping Ceph mds.f for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:25:16.289 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:25:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f[65067]: 2024-09-11T22:25:15.805+0000 7fa45bc12640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:25:16.289 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:25:15 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f[65067]: 2024-09-11T22:25:15.805+0000 7fa45bc12640 -1 mds.f *** got signal Terminated *** 2024-09-11T22:25:18.948 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:25:18 smithi153 podman[70867]: 2024-09-11 22:25:18.679118442 +0000 UTC m=+2.976813432 container died 9c7c2db94ea0492648e76835b98daf9782cbead66e1142c10fa81f44e1e1248d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T22:25:19.289 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:25:19 smithi153 podman[70867]: 2024-09-11 22:25:19.047665757 +0000 UTC m=+3.345360746 container cleanup 9c7c2db94ea0492648e76835b98daf9782cbead66e1142c10fa81f44e1e1248d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T22:25:19.289 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 11 22:25:19 smithi153 bash[70867]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-f 2024-09-11T22:25:19.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T22:25:19.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:25:19.510 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T22:25:19.510 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T22:25:19.510 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.i 2024-09-11T22:25:19.827 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:25:19 smithi153 systemd[1]: Stopping Ceph mds.i for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:25:20.289 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:25:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i[65674]: 2024-09-11T22:25:19.826+0000 7f4715468640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:25:20.289 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:25:19 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i[65674]: 2024-09-11T22:25:19.826+0000 7f4715468640 -1 mds.i *** got signal Terminated *** 2024-09-11T22:25:22.789 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:25:22 smithi153 podman[70988]: 2024-09-11 22:25:22.68224749 +0000 UTC m=+2.959363941 container died f9f398948c24a9a3834d3a74141bd9904a3fec02855f06a4bb812df3d9a77640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T22:25:23.289 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:25:23 smithi153 podman[70988]: 2024-09-11 22:25:23.059523223 +0000 UTC m=+3.336639663 container cleanup f9f398948c24a9a3834d3a74141bd9904a3fec02855f06a4bb812df3d9a77640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T22:25:23.289 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 11 22:25:23 smithi153 bash[70988]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-i 2024-09-11T22:25:23.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T22:25:23.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:25:23.468 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T22:25:23.469 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T22:25:23.469 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.l 2024-09-11T22:25:23.780 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:25:23 smithi153 systemd[1]: Stopping Ceph mds.l for a6709e28-7089-11ef-bcea-c7b262605968... 2024-09-11T22:25:24.038 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:25:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l[64762]: 2024-09-11T22:25:23.779+0000 7f0cda76c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:25:24.039 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:25:23 smithi153 ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l[64762]: 2024-09-11T22:25:23.779+0000 7f0cda76c640 -1 mds.l *** got signal Terminated *** 2024-09-11T22:25:27.022 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:25:26 smithi153 podman[71110]: 2024-09-11 22:25:26.681979359 +0000 UTC m=+2.998381127 container died d1a8ee70b076e97c301cea4c18d800b416704d6701bb3bdd9496a31345f1be65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T22:25:27.022 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:25:27 smithi153 podman[71110]: 2024-09-11 22:25:27.021676487 +0000 UTC m=+3.338078254 container cleanup d1a8ee70b076e97c301cea4c18d800b416704d6701bb3bdd9496a31345f1be65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T22:25:27.289 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:25:27 smithi153 bash[71110]: ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l 2024-09-11T22:25:27.289 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 11 22:25:27 smithi153 podman[71122]: 2024-09-11 22:25:27.222862282 +0000 UTC m=+0.539711921 container remove d1a8ee70b076e97c301cea4c18d800b416704d6701bb3bdd9496a31345f1be65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-a6709e28-7089-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T22:25:27.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6709e28-7089-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T22:25:27.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:25:27.472 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T22:25:27.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6709e28-7089-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:25:27.673 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:25:30.294 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6709e28-7089-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:25:30.493 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:25:32.951 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6709e28-7089-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:25:33.153 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:25:35.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:25:35.561 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:25:35.595 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:25:35.621 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T22:25:35.623 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi092/crash 2024-09-11T22:25:35.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:25:35.651 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:25:35.651 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:25:35.653 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi110/crash 2024-09-11T22:25:35.654 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:25:35.706 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:25:35.706 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:25:35.708 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi153/crash 2024-09-11T22:25:35.709 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:25:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/a6709e28-7089-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:25:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:25:35.746 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T22:25:35.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-11T22:25:35.791 INFO:tasks.cephadm:Compressing logs... 2024-09-11T22:25:35.791 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:25:35.793 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:25:35.796 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:25:35.816 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-11T22:25:35.816 INFO:teuthology.orchestra.run.smithi092.stderr:: No such file or directory 2024-09-11T22:25:35.818 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T22:25:35.818 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T22:25:35.818 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.log: /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:25:35.818 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:25:35.818 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T22:25:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:25:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose 92.6% -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T22:25:35.821 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-11T22:25:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T22:25:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.a.log: 82.7% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:25:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T22:25:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:25:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:25:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T22:25:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-09-11T22:25:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T22:25:35.827 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T22:25:35.827 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:25:35.827 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T22:25:35.827 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T22:25:35.827 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:: No such file or directory 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T22:25:35.830 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 82.8% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 90.5% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.l.log: /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log: 86.6% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:25:35.831 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T22:25:35.833 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:25:35.833 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.4.log: /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:25:35.833 INFO:teuthology.orchestra.run.smithi110.stderr: 90.4% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:25:35.833 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T22:25:35.833 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:25:35.834 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:25:35.836 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:25:35.836 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T22:25:35.837 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T22:25:35.837 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log: 86.6% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:25:35.837 INFO:teuthology.orchestra.run.smithi092.stderr: 91.0% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:25:35.837 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T22:25:35.837 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T22:25:35.840 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.x.log: 94.7% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T22:25:35.840 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T22:25:35.840 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.y.log: 92.7% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T22:25:35.840 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T22:25:35.840 INFO:teuthology.orchestra.run.smithi110.stderr: 90.8% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:25:35.841 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.z.log: 92.7% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T22:25:35.842 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log: 86.6% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:25:35.842 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T22:25:35.844 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.8% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T22:25:35.845 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-11T22:25:35.846 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T22:25:35.846 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T22:25:35.850 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:25:35.852 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:25:35.864 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:25:35.873 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:25:35.887 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:25:35.888 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:25:36.079 INFO:teuthology.orchestra.run.smithi092.stderr: 90.4% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T22:25:36.157 INFO:teuthology.orchestra.run.smithi153.stderr: 91.9% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T22:25:36.342 INFO:teuthology.orchestra.run.smithi110.stderr: 92.0% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T22:25:36.853 INFO:teuthology.orchestra.run.smithi092.stderr: 90.5% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T22:25:37.877 INFO:teuthology.orchestra.run.smithi110.stderr: 93.9% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T22:25:38.051 INFO:teuthology.orchestra.run.smithi092.stderr: 94.1% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T22:25:38.170 INFO:teuthology.orchestra.run.smithi092.stderr: 94.0% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T22:25:38.378 INFO:teuthology.orchestra.run.smithi110.stderr: 94.2% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T22:25:40.577 INFO:teuthology.orchestra.run.smithi153.stderr: 94.7% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T22:25:44.079 INFO:teuthology.orchestra.run.smithi092.stderr: 91.7% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T22:25:44.916 INFO:teuthology.orchestra.run.smithi110.stderr: 91.5% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T22:25:45.212 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T22:25:45.244 INFO:teuthology.orchestra.run.smithi153.stderr: 91.2% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T22:25:45.633 INFO:teuthology.orchestra.run.smithi110.stderr: 91.6% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T22:25:46.134 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T22:25:46.348 INFO:teuthology.orchestra.run.smithi153.stderr: 91.1% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T22:25:47.657 INFO:teuthology.orchestra.run.smithi092.stderr: 91.4% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T22:25:47.858 INFO:teuthology.orchestra.run.smithi110.stderr: 91.3% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T22:25:48.636 INFO:teuthology.orchestra.run.smithi110.stderr: 91.1% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T22:25:49.423 INFO:teuthology.orchestra.run.smithi153.stderr: 91.2% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T22:25:49.426 INFO:teuthology.orchestra.run.smithi153.stderr: 91.3% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T22:26:07.549 INFO:teuthology.orchestra.run.smithi110.stderr: 94.1% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T22:26:13.057 INFO:teuthology.orchestra.run.smithi153.stderr: 94.1% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T22:26:14.174 INFO:teuthology.orchestra.run.smithi110.stderr: 94.3% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T22:26:14.176 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-11T22:26:14.176 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m38.365s 2024-09-11T22:26:14.176 INFO:teuthology.orchestra.run.smithi110.stderr:user 1m54.616s 2024-09-11T22:26:14.176 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m3.589s 2024-09-11T22:26:14.364 INFO:teuthology.orchestra.run.smithi153.stderr: 94.3% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T22:26:44.512 INFO:teuthology.orchestra.run.smithi153.stderr: 93.5% -- replaced with /var/log/ceph/a6709e28-7089-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T22:26:44.513 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-09-11T22:26:44.513 INFO:teuthology.orchestra.run.smithi153.stderr:real 1m8.703s 2024-09-11T22:26:44.514 INFO:teuthology.orchestra.run.smithi153.stderr:user 3m10.119s 2024-09-11T22:26:44.514 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m5.826s 2024-09-11T22:28:11.369 INFO:teuthology.orchestra.run.smithi092.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-11T22:28:11.372 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-11T22:28:11.372 INFO:teuthology.orchestra.run.smithi092.stderr:real 2m35.563s 2024-09-11T22:28:11.372 INFO:teuthology.orchestra.run.smithi092.stderr:user 3m14.491s 2024-09-11T22:28:11.373 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m6.103s 2024-09-11T22:28:11.373 INFO:tasks.cephadm:Archiving logs... 2024-09-11T22:28:11.375 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi092/log 2024-09-11T22:28:11.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T22:28:45.338 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi110/log 2024-09-11T22:28:45.339 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T22:29:01.442 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi153/log 2024-09-11T22:29:01.443 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T22:29:27.703 INFO:tasks.cephadm:Removing cluster... 2024-09-11T22:29:27.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6709e28-7089-11ef-bcea-c7b262605968 --force 2024-09-11T22:29:27.907 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:29:28.279 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6709e28-7089-11ef-bcea-c7b262605968 --force 2024-09-11T22:29:28.480 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:29:28.921 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6709e28-7089-11ef-bcea-c7b262605968 --force 2024-09-11T22:29:29.124 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: a6709e28-7089-11ef-bcea-c7b262605968 2024-09-11T22:29:29.599 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T22:29:29.600 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T22:29:29.619 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T22:29:29.635 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T22:29:29.651 INFO:tasks.cephadm:Teardown complete 2024-09-11T22:29:29.651 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T22:29:29.663 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T22:29:29.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T22:29:29.668 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T22:29:29.678 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T22:29:29.786 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T22:29:29.797 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T22:29:29.798 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:29:29.801 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:29:29.803 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:29:29.814 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:29:29.816 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:29:29.818 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:29:29.821 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:29:29.821 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-11T22:29:29.821 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1045us[+1039us] +/- 48ms 2024-09-11T22:29:29.821 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -195us[ -195us] +/- 67ms 2024-09-11T22:29:29.821 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -1442us[-1449us] +/- 75ms 2024-09-11T22:29:29.821 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:29:29.822 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:29:29.822 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-11T22:29:29.822 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +1017us[+1029us] +/- 48ms 2024-09-11T22:29:29.822 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -215us[ -215us] +/- 67ms 2024-09-11T22:29:29.822 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -1455us[-1455us] +/- 75ms 2024-09-11T22:29:29.822 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:29:29.823 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:29:29.823 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-11T22:29:29.823 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +1035us[+1061us] +/- 48ms 2024-09-11T22:29:29.823 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 -193us[ -168us] +/- 67ms 2024-09-11T22:29:29.823 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -1444us[-1444us] +/- 75ms 2024-09-11T22:29:29.823 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:29:29.823 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T22:29:29.834 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T22:29:29.835 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T22:29:29.844 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T22:29:29.939 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T22:29:30.005 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T22:29:30.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:29:30.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:29:30.101 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:29:30.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:29:30.131 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:29:30.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:29:30.159 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T22:29:30.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T22:29:30.178 INFO:teuthology.task.internal:Duration was 2078.421203 seconds 2024-09-11T22:29:30.178 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T22:29:30.187 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T22:29:30.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T22:29:30.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T22:29:30.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T22:29:30.226 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:29:30.228 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:29:30.238 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:29:30.560 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T22:29:30.560 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T22:29:30.560 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T22:29:30.621 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T22:29:30.621 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T22:29:30.649 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-09-11T22:29:30.650 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T22:29:30.678 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T22:29:30.678 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T22:29:30.680 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T22:29:30.692 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T22:29:30.745 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T22:29:30.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T22:29:30.747 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T22:29:30.758 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T22:29:31.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T22:29:31.284 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T22:29:31.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T22:29:31.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T22:29:31.335 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T22:29:31.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T22:29:31.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:29:31.374 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:29:31.378 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:29:31.395 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-11T22:29:31.398 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-11T22:29:31.425 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-09-11T22:29:31.464 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:29:31.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:29:31.479 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:29:31.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:29:31.494 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:29:31.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:29:31.519 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T22:29:31.530 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T22:29:31.531 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi092 2024-09-11T22:29:31.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T22:29:31.577 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi110 2024-09-11T22:29:31.577 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T22:29:31.613 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901413/remote/smithi153 2024-09-11T22:29:31.613 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T22:29:31.649 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T22:29:31.650 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T22:29:31.652 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T22:29:31.655 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T22:29:31.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T22:29:31.712 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T22:29:31.712 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T22:29:31.722 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T22:29:31.722 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T22:29:31.725 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T22:29:31.727 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T22:29:31.737 INFO:teuthology.orchestra.run.smithi092.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 22:29 /home/ubuntu/cephtest 2024-09-11T22:29:31.739 INFO:teuthology.orchestra.run.smithi110.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 22:29 /home/ubuntu/cephtest 2024-09-11T22:29:31.753 INFO:teuthology.orchestra.run.smithi153.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 22:29 /home/ubuntu/cephtest 2024-09-11T22:29:31.755 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T22:29:31.765 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T22:29:31.790 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/blogbench}} duration: 2078.421203136444 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-11T22:29:31.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T22:29:31.887 INFO:teuthology.run:pass